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FOREWORD 


In  a  letter  under  the  date  of  2  November  1967,  Dr.  John  L. 

McDaniel,  Technical  Director  of  the  Research  and  Engineering 
Directorate  at  the  U.  S.  Army  Missile  Command  (MICOM),  offered  to 
hold  the  Fourteenth  Conference  on  the  Design  of  Experiments  in 
Army  Research,  Development  and  Testing  at  his  installation.  Since 
arrangements  were  already  underway  to  hold  this  conference  in  the 
Washington  area,  this  invitation  had  to  be  declined  by  the  Army 
Mathematics  Steering  Committee  (AMSC) ,  the  sponsor  of  this  series 
of  conferences.  Dr.  McDaniel,  when  made  aware  of  this  situation, 
was  willing  for  the  Committee  to  treat  his  request  to  hold  the 
conference  as  a  standing  Invitation.  Members  of  the  AMSC  were  very 
pleased  to  hear  this  and  then  discussed  with  him  the  possibility  of 
holding  the  Fifteenth  Conference  at  Redstone  Arsenal.  These  negotia¬ 
tions  were  brought  to  a  successful  conclusion;  and,  on  29  November  1968, 
Major  General  Charles  W.  Eifler  Issued  a  formal  invitation  to  host  this 
conference  at  his  command  on  22-24  October  1969.  He  appointed  Dr.  Siegfried 
Lehnigk  to  serve  as  Chairman  on  Local  Arrangements  and  Mr.  Raymond  V.  Knox 
to  handle  administrative  requirements. 

MICOM  had  already  served  as  the  host  to  the  Ninth  Conference  in 
this  series.  It  is  interesting  to  note  that  Dr.  Lehnigk,  as  well  a3 
Henry  A.  Dlhm,  and  W.  H.  Ewart  served  as  members  of  the  Local  Arrange¬ 
ments  Committee  for  the  Ninth  Conference,  as  well  as  the  Fifteenth 
Conference.  Those  in  attendance  at  this  22-24  October  meeting  are 
much  in  debt  to  these  gentlemen,  as  well  as  to  many  others  at  Redstone 
Arsenal,  for  the  excellent  handling  of  the  many  details  connected  with 
a  meeting  of  this  size. 

Among  the  many  highlights  of  the  Fifteenth  Conference  on  the  Design 
of  Experiments  was  the  banquet  talk  given  by  Professor  Oskar  Morgenstern 
of  Princeton  University  and  the  following  invited  speakers: 

Reliability  Applied  to  Space  Flight 

Dr.  John  E.  Condon,  National  Aeronautics  and 
Space  Administration 

Systems  Reliability 

Dr.  Nancy  R.  Mann,  Rocketdyne 

A  Probability  Approach  to  Catastrophic  Threat 

Dr.  Clifford  J.  Maloney,  National  Institutes  of  Health 

The  Empirical  Bayes  Approach  to  the  Design  and  Analysis 

of  Experiments 

Professor  Richard  G.  Krutchkoff,  Virginia  Polytechnic 
Institute 
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On  Confidence  Limits  for  the  Performance  o£  a  System  When 

Few  Failures  jre  Encountered 

Dr.  S.  C.  Saunders,  Boeing  Scientific  Research  Laboratories 

Everyone  had  the  opportunity  to  hear  the  above-mentioned  talks,  as  they 
were  given  in  general  sessions.  Unfortunately,  one  was  not  privileged 
to  hear  all  of  the  thirty-two  contributed  papers.  These  covered  a  wide 
range  of  interesting  statistical  problems  and  had  to  be  scheduled  so 
that  three  talks  were  conducted  simultaneously.  Following  the  banquet, 
it  waG  my  privilege  to  award  the  Fifth  Samuel  3.  Wilks  Memorial  Medal, 
sponsored  by  the  American  Statistical  Association  and  the  Army,  to 
Dr.  W.  J.  YouJen.  Details  of  this  presentation  are  included  in  these 
Proceedings. 

This  conference  was  attended  by  156  scientists;  and  52  organizations 
were  represented.  Speakers  and  panelists  came  from:  Boeing  Scientific 
Research  Laboratories;  Cornell  University;  Honeywell,  Inc.;  Litton 
Systems,  Inc.;  National  Aeronautics  and  Space  Administration;  National 
Institutes  of  Health;  North  Carolina  State  University;  Princeton 
University;  Rocketdyne;  University  of  Alabama;  University  of  Georgia; 
University  of  Michigan;  University  of  Wisconsin;  Vanderbilt  University; 
Virginia  Polytechnic  Institute;  and,  12  Army  facilities. 

Members  of  the  AMSC  would  like  to  express  their  thanks  to  the 
many  speakers,  chairmen  and  panelists  for  all  their  efforts  in  behalf 
of  this  important  scientific  meeting.  Most  of  the  papers  presented  at 
the  conference  are  being  made  available  to  the  public  through  these 
Proceedings.  The  AMSC  asked  that  cbpies  of  this  manual  receive  wide 
distribution  among  Army  laboratories  and  Technical  Libraries. 

At  this  time,  let  me  express  my  appreciation  to  all  members  of  the 
Program  Committee  (Clifford  Cohen,  Jr.,  Henry  Dihm,  Francis  Dressel, 
Walter  Foster,  Fred  Frishman,  Bernard  Harris,  Boyd  Harshbarger,  Raymond 
Knox,  Siegfried  Lehnigk,  H.  L.  Lucas,  Clifford  Maloney,  and  Herbert 
Solomon)  for  their  many  suggestions  and  advice  on  the  selection  of  the 
speakers  and  the  organization  of  the  whole  conference. 


Frank  E.  Grubbs 
Conference  Chairman 
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RELIABILITY  APPLIED  TO  SPACE  FLIGHT 

John  E.  Condon 
NASA  Headquarters 
Washington,  D.  C. 


This  month,  October,  marks  the  eleventh  anniversary  of  NASA. 
Reflecting  on  NASA's  accomplishments  during  the  past  eleven  years,  I 
feel  we  can  point  with  pride  to  an  outstanding  record  cf  success.  Our 
record  of  mission  success  during  these  eleven  years  is  over  75%,  topped 
by  a  manned  flight  record  of  outstanding  success  in  the  Mercury,  Gemini 
and  Apollo  programs. 

The  superlatives  have  been  exhausted  in  describing  the  success 
and  significance  of  Apollo  -  particularly  Apollo  11.  I  suspect  that 
many  of  you  are  keenly  interested  in  knowing  how  we  have  attained  the 
level  of  reliability  so  vital  to  the  success  of  the  Apollo  program.  I 
have  given  a  great  deal  of  thought  to  this  subject  during  the  past 
three  months  and  regretfully  -  though  not  unexpectedly  -  have  not  found 
a  simple,  concise  answer  to  this  question.  There  are  many  factors  which 
have  contributed  to  the  reliability  of  Apollo  and  thus  it  is  not  possible 
to  single  out  any  one  factor  as  being  all  encompassing.  However,  there 
are  two  areas  which,  in  my  view,  are  worthy  of  special  attention: 

].  major  attention  by  top  management  to  the  reliability  of 
Apollo  hardware;  and, 

2.  emphasis,  through  all  phases  of  the  program,  on  the 
engineering  aspects  of  reliability. 

I  will  devote  my  remarks  to  the  latter  of  these  two  points  following  some 
brief  comments  on  the  former. 

The  effective  attainment  of  reliable  space  hardware  requires  the 
attention  of  all  members  of  program/project  team  coupled  with  strong 
management  support.  This  has  been  a  key  factor  in  the  success  of  Apollo 
as  top  management  has  actively  participated  in  key  milestone  reviews 
which  are  so  important  to  the  successful  performance  of  the  system.  To 
illustrate  this  point,  the  following  are  examples  of  key  Apollo  milestone 
reviews . 

Critical  Design  Review.  The  purpose  of  this  review  is  to  formally 
review  the  design  of  the  Contract  End  Item  when  the  design  is 
essentially  complete.  The  review  is  Intended  to  precede  the 
release  of  engineering  for;  manufacture.  Among  other  things, 
this  review  established  the  integrity  of  the  design  by  review 
of  analytical  and  test  data,  and  reliability  apportionment  and 
analysis  available  at  that  particular  point  in  time. 
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Certification  of  Flight  Worthiness.  The  purpose  of  this 
uillea  luiic  i  ci  ic-  cor  1  ify  that  each  flight  6 1  ag*?  mnHiil# 

is  a  complete  and  qualified  item  of  hardware  prior  to  ship¬ 
ment  and  is  accompanied  by  adequate  and  accurate  supporting 
documentation.  Through  this  review  the  Apollo  Program 
Director  is  informed  of  any  deficiencies  prior  to  shipment 
of  the  stage  or  module.  This  review  certifies,  for  example, 
that: 

1.  acceptance,  qualification  and  reliability  tests 
have  been  successfully  completed  and  meet  the 
specification  requirements; 

2.  departures  from  specification  and  drawing  requirements 
have  been  approved  by  Material  Review  Boards; 

3.  critical  hardware  failures  have  been  analyzed  and 
corrected. 

Flight  Readiness  Review  (FRR),  This  is  a  two  part  review 
scheduled  for  each  mission  by  a  joint  letter  signed  by  the 
Program  Director  and  the  Mission  Director.  The  purpose  of 
the  Program  Director's  FRR  is  to  determine  that  the  space 
vehicle  hardware:  and  launch  complex  are  ready  to  commence 
the  mission  period.  This  includes  consideration  of  the  check¬ 
out  and  qualification  status  of  all  hardware,  the  summary  of 
failures  and  disposition  thereof,  with  particular  emphasis  on 
failures  that  have  occurred  during  the  pre-launch  and  checkout 
phase,  and  all  modifications,  deviations  and  waivers.  The 
purpose  of  the  Mission  Director's  FRR  is  to  make  a  judgment 
for  initiating  the  mission  period  and  committing  the  deployment 
of  world-wide  forces  to  support  the  mission.  Upon  satisfactory 
completion  of  the  Flight  Readiness  Review  the  mission  period 
will  commence. 

The  active  participation  of  top  management  in  these  reviews  gives 
emphasis  to  their  importance,  helps  ensure  that  all  factors  which  influence 
the  successful  performance  of  the  hardware  have  received  proper  attention, 
and  results  In  a  "team"  approach  to  system  reliability. 

The  nature  of  NASA  systems  -  highly  complex,  small  quantity,  R4D 
systems  -  requires  that  we  concentrate  on  the  engineering  aspects  of 
reliability  rather  than  the  analytical  aspects,  particularly  at  the 
system  and  major  subsystem  levels.  In  this  regard,  I  would  like  to 
discuss  the  following: 

1.  adequacy  of  design  for  mission  requirements; 

2.  identification  and  control  of  failure  modes; 


2 


3.  testing;  and,  . 


4.  identification  and  correction  of  ail  failures. 

We  place  heavy  emphasis  on  the  design  review  function  and  require 
our  contractors,  as  part  of  their  reliability  program,  to  have  a  design 
review  program.  Contractors  are  required  to  establish  and  conduct  a 
formal  program  of  planned,  scheduled  and  documented  design  reviews  at 
the  system,  subsystem  and  component  levels.  These  reviews  are  compre¬ 
hensive  critical  audits  of  all  pertinent  aspects  of  the  design  of  the 
hardware  and  software  and  are  conducted  at  major  program  milestones 
beginning  in  the  feasibility  stage.  Participation  in  these  design 
reviews  should  be  Inter-organizational  including  competent  personnel 
from  such  areas  as  design,  fabrication,  test,  reliability  assurance, 
quality  assurance,  and  parts  applications.  In  this  way,  inter¬ 
disciplinary  engineering  competence  is  brought  to  bear  on  all  aspects 
of  hardware  design  so  as  to  identify  and  eliminate  potential  problems. 

NASA  personnel  may  participate  in  these  design  reviews  as  deemed  neces¬ 
sary.  Each  design  review  must  be  documented  and  the  contractor's 
reliability  organization  is  responsible  for  follow-up  action  to  ensure 
that  all  recommendations  are  satisfactorily  completed.  An  effective  ' 
design  review  program  pays  high  dividends  through  the  early  identification 
and  elimination  of  problems  which  would  manifest  themselves  at  a  later 
time  when  correction  may  be  more  costly. 

Also,  as  an  integral  part  of  the  early  design  phase,  we  require  the 
contractor  to  develop  analyses  to  determine  possible  modes  of  failure 
and  their  effects  on  mission  objectives  and  crew  safety.  These  analyses 
are  conducted  at  the  system,  subsystem  and  component  levels.  Each  potential 
failure  is  considered  in  terms  of  its  probability  of  occurrence  and  is 
categorized  as  to  probable  effect  on  mission  success;  e.g.,  loss  of  life 
of  crewmember,  mission  termination,  launch  scrub  or  delay,  etc.  These 
analyses,  generally  referred  to  as  Failure  Mode,  Effect  and  Criticality 
Analyses  (FMEA)  have  the  following  important  applications : 

1.  determining  the  need  for  redundancy,  fail-safe  design  and 
derating; 

2.  determining  the  need  to  select  parts  and  components  of 
higher  reliability; 

3.  identifying  single  failure  points  and  reducing  such  to 
acceptable  levels  of  risk; 

4.  supporting  reliability  predictions  and  assessments; 

5.  supporting  system  safety  and  hazard  analyses; 

6.  assuring  that  test  programs  are  responsive  to  known  and 
suspected  potential  failure  lodes; 
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7.  establishing  allowing  operating  times  or  cycles;  and, 

8.  determining  operational  contingency  plans. 

% 

0£  particular  importance  in  out'  manned  flight  program' is  the  u&e  of 
FMEA's  to  identify  single  failure  points  which  could  adversely  effect 
crew  safety  and  mission  objectives. 

f 

NASA  places  strong  emphasis  on  testing  throughout  all  phases  of 
hardware  development  and  fabrication.  We  require  the  contractor  t. 
develop  an  integrated  test  program  which  will  evaluate  all  aspects  ct 
system  performance  capability  to  the  extent  practical.  In  terms  c< 
reliability  considerations  we  expect  the  testing  program  to  be  di reeled 
towards : 

1.  verifying  the  capability  of  the  design; 

2.  evaluating  the  susceptibility  of  the  design  and  hardware 
to  failures; 

3.  identifying  unexpected  interactions  among  components  and 
assemblies; 

4.  identifying  failure  modes  which  reflect  defects  in  materials, 
workmanship  and  fabrication  processes;  and, 

5.  obtaining  failure  rate  and  other  reliability  data. 

i 

To  the  extent  practical,  tests  are  planned  using  statistical  design- 
of-experlment  techniques  and  are  conducted  under  environmental  conditions 
and  for  time  periods  commensurate  with  mission  conditions. 

The  final  area  to  be  discussed  is  that  of  failure  reporting  and 
corrective  action.  We  expect  all  failures  and  nonconformances. to  be 
identified,  analysed  and  effective  correction  action  taken  -  we  cannot 
tolerate  unexplained  failures  .r  ineffective  corrective  action  in  our 
space  programs.  We  specifically  require  our  contractors  to  employ  a 
controlled  system  for  identification,  reporting,  analysis,  correction 
and  prevention  of  recurrence  of  all  nonconformances  and  suspected  non¬ 
conformance  of  a  functional  nature  which  occur  throughout  the  contract 
period.  Some  of  the  requirements  which  the  system  must  satisfy  are  as 
f ol lows : 

1.  it  shall  cover  hardware,  tertain  software,  the  interfaces 
between  hardware  and  software  and  the  interfaces  between 
hardware  or  software  and  test  or  operational  personnel; 

2.  Lt  shall  cover  all  nonconformances  or  suspected  nonconformances 
of  a  functional  nature  such  as; 
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a.  unusual  condition  occurring  in  test  or  handling  which 
are  uuspected  to  have  an  effect  on  the  hardware; 

b.  transient  malfunctions  and  suspected  malfunctions;  and, 

c.  notable  deviations  from  previous  performance  -  parameter 
drift. 

3.  It  shall  provide  for  investigation  of  each  reported  failure 
by  an  engineering  analyses,  followed,  where  appropriate,  by 
laboratory  analysis  of-  failed  hardware.  Such  investigation 
shall  be  adequate  to  assess  causes,  mechanisms,  and  potential 
effects  of  the  failure  and  serve  as  a  basis  for  decisions  or. 
the  most  efficient  remedial  and  preventive  actions; 

4.  it  shall  provide  for  a  review  of  the  technical  closeout 
decision  on  each  reported  failure  by  higher  levels  of 
technical  management  commensurate  with  the  criticality 
category  of  the  failure  involved;  and, 

5.  closeout  action  shall  be  considered  complete  when: 

a.  remedial  actions  have  been  accomplished ; 

necessary  preventive  design  and  Software  changes  have 
been  devised  and  accomplished; 

necessary  design  or  computer  program  changes  have  been 
verified  in  test; 

/ 

effectivity  of  preventive  actions  have  been  established; 

change  has  been  made  in  existing  identical  items  of 
hardware  to  which  the  change  is  pertinent;  and, 

closeout  documentation  has  been  signed  by  proper  management 
authority. 

Such  a  system  may  seem  unnecessarily  extensive  but  experience  has  shown 
that  it  is  necessary  and  pays  high  dividends.. 

In  conclusion,  I  would  like  to  point  out  that  a  significant  portion 
of  our  reliability  problems  are  due  to  nonelectronic  parts  and  components. 

Such  items  as  valves,  fittings,  seals,  actuators,  etc.,  continue  to  receive 
major  attention  as  we  strive  to  attain  the  levels  of  reliability  necessary 
for  mission  success. 

As  we  look  to  the  future  wa  will  be  striving  to  decrease,  significantly, 
our  cost  per  pound  of  payload,  the  complexity  of  our  systems  will  continue  to 
increase  and,  thus,  our  need  for  strong  emphasis  on  the  engineering  aspects 
of  reliability  will  not  abate. 


b. 

c. 

d. 

e. 

f. 
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MONTE  CARLO  CONFIDENCE  BOUNDS  ON  SYSTEM  RELIABILITY* 

Nancy  R.  Mann 
Rocketdyne 

Canoga  Park,  California 


ABSTRACT .  A  description  is  given  of  results  of  preliminary 
investigations  (by  a  group  at  North  American  Rockwell  Corporation)  re¬ 
lated  to  the  Monte  Carlo  generation  of  lower  confidence  bounds  on  the 
reliability  of  a  logically  complex  system.  In  calculating  system  confidence 
bounds  by  use  of  a  Monte  Carlo  procedure,  one  must  generate  the  distribution 
of  each  Independent  subsystem  reliability,  given  the  life-test  failure  data 
for  that  subsystem.  Therefore,  an  assumption  of  a  specified  a  priori  dis¬ 
tribution  for  each  subsystem  reliability  is  implicit  in  the  procedure. 

In  order  that  clues  may  be  obtained  as  to  optimum  prior  assumptions 
to  be  used  in  calculating  Monte  Carlo  bounds  for  a  complex  system,  the 
model  has  been  restricted  to  a  series  system  wherein  each  independent 
subsystem  has  exponentially  distributed  failure  time  and  prototypes  of 
each  subsystem  are  tested  until  a  fixed  (but  not  necessarily  the  same  for 
each  subsystem)  number  of  failures  occurs.  For  this  model,  optimum 
(uniformly  most  accurate  unbiased)  exact  classical  confidence  bounds  on 
the  reliability  R(t  )  at  a  specified  mission  time  t  are  available, 

although  not  easily  calculated  (Lentner,  M.  M  and  Buehler,  R.  J. ,  1963. 

J.  Ainer.  Statist.  Assoc,  58.  670-677  and  El  Mawaziny,  A.  H. ,  1965.  Un¬ 
published  doctoral  dissertation,  Iowa  State  University).  Computer  programs 
for  calculating  the  optimum  classical  bounds  and  the  Bayesian  Monte  Carlo 
bounds  were  written,  and  a  means  of  numerically  comparing  various  forms 
of  prior  distributions  against  an  optimum  standard  was  thus  provided. 

One  prior  distribution  widely  used  in  obtaining  Monte  Carlo  and  general 
Bayesian  exact  lower  aonfidenae  bounds  on  system  reliability  is  thereby 
shewn  numerically  to  yield  bounds  whioh  are  (conservative  in  the  classical 
sense  for  this  series-system  model.  Another  suggested  prior  distribution 
is  shown  to  give  bounds  which  are  usually  conservative  but  under  certain 
conditions  are  liberal,  and  henoe  not  truly  confidence  bounds.  Moreover, 
it  is  demonstrated  by  a  combination  of  numerical  and  analytical  results, 
that  for  a  series  system  containing  more  than  one  independent  subsystem 


*This  research  was  sponsored  by  the  Mathematics  and  Statistics  Panel  of 
the  Aerospace  and  Systems  Group  (A&SG)  of  North  American  Rockwell 
Corporation  and  funded  under  the  Internal  Research  and  Development 
program  of  the  Executive  Offices  of  A&SG. 
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there  exists  no  prior  distribution  for  subsystem  reliability  whiah  is 
independent-  of  the  'into.  and  which  yields  the  optimum  lower  hounds. 

Other  numerical  results  reLated  to  the  selection  of  optimum  methods 
for  generating  the  bounds  and  evaluation  of  certain  approximate  methods 
are  described. 


BACKGROUND  AND  APPROACH 

Review  of  Pertinent  Literature.  If  it  is  possible  to  determine 
confidence  bounds  on  system  reliability  solely  from  the  testing  of  the 
subsystems  of  which  the  system  is  comprised,  saving  of  expensive  system 
testing  can  be  effected.  It  may,  in  fact,  sometimes  be  Infeasible  to 
test  the  system  as  a  whole.  Furthermore,  this  method  of  obtaining 
system  confidence  bounds  can  be  used  for  exploratory  system  design. 

The  subject  of  conildence  bounds  for  system  reliability  from 
subsystem  testing  is  one  about  which  much  has  been  written,  but  not  a 
great  deal  is  known.  Consider  a  series  system  in  which  the  failure  times 
of  k  independent  subsystems  are  exponentially  distributed;  i.e.t  for  T 
a  random  variable  representing  failure  time,  Prob  (T>t)  a  R(t)  ■  exp(-Xt), 
t>0,  \>0.  Suppose  prototypes  of  the  jth  subsystem,  are  subjected  to 

life  test  and  the  life  test  is  terminated  at  the  time  of  the  r.th  ordered 

~3 

failure,  j-1 , 2 , . . . ,k.  For  this  special  model,  there  exist  optimum 
(uniformly  most  accurate  unbiased)*  exact^  confidence  bounds  on  the 
reliability  R(tm)  at  time  t^,  the  probability  that  the  system  will 

survive  at  least  until  time  t  .  [See  Lentner  and  Buehler  (27)  and  El 

to 

Mawazlny  (12)].  No  such  optimum  bounds  have  been  found  for  a  model 
which  is  equivalent  to  this  exponential-failure-time  series-system  model, 
except  for  the  fact  that  total  test  time  tj  rather  than  number  of  failures 

Tj  is  specified  for  the  life  test  of  Jth  subsystem,  j»l,2,...,k,  and 

number  of  failures  is  the  observable  random  variable.  For  either  the 
fixed-time  or  fixed-number  of  failures  model,  optimum  exact  confidence 


*The  definitions  of  uniformly  most  accurate  and  unbiased  confidence  bounds 
are  as  given  by  E.  Lehman  (26).  They  are  as  follows:  A  confidence  bound 
£(X)  satisfying  Pp  (jKX)^e)  ^  1-a  for  all  0  and  for  all  0'<0,  P0{0_(X)£0 1  )s 
minimum  is  a  uniformly  most  accurate  lower  confidence  bound  for  0  at  level 
1-a  .  A  family  of  lower  confidence  bounds  at  level  l-ct  ia  said  to  ba  un¬ 
biased  if  Pi}{£(X)^9'  }<.  l-«  for  all  O'*0  for  all  0. 

‘A  lower  confidence  bound  at  level  1-a  is  said  to  be  exact  if 
pi((£(X)^'.' 1  *  1-..  for  all  0. 
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bounds  have  not  been  derived  for  cases  in  which  either  failure  time  has 
utiie l  than  an  exponential  distribution  (or  can  be  converted  by  a  trans¬ 
formation  of  the  data  to  an  exponential  distribution)  or  the  system  is 
other  than  an  independent  series  system. 

Another  much  used  failure  model,  often  called  the  "attribute” 
model,  is  one  in  which  only  pass-fail  binomlally  distributed  data  are 
collected  for  each  independent  subsystem.  For  this  model,  optimum  exact 
confidence  bounds  on  reliability  (or  probability  of  successful  operation) 
of  a  series  system  have  been  derived  [see  Buehler  (6)],  but  the  problem 
of  actually  constructing  such  optimum  bounds  has  not  beep  completely 
solved  [see  Lipow  (31),  Lipow  (32),  Lloyd  and  Lipow  (33),  Steck  (48), 
and  Schick  (43)].  If  a  Poisson  approximation  to  the  binomial  distribution 
is  applicable,  then  results  of  Harris  (22)  provide  optimal  exact  bounds 
on  the  reliability  of  an  independent  series  system  for  the  attribute, 
model  if  one  randomizes  appropriately  in  obtaining  the  bounds.  One 
would  expect  the  Poisson  approximation  to  the  binomial  distribution  to 
apply  when  the  number  of  prototypes  of  each  subsystem  tested  is  large 
and  the  probability  of  failure  for  each  subsystem  is  small.  There  ap¬ 
pears  to  be  some  question,  however,  [see  Garner  (19))  as  to  whether  the 
approximation  loses  its  applicability  as  the  number  of  subsystems  increases. 

Many  approximate  and  non-optimal  exact  confidence  bounds  on  system 
reliability  have  been  derived.  There  have  been  several  approximate  con¬ 
fidence  bounds  on  system  reliability  at  time  t^  derived  for  the  exponential 

f ixed-number-of-f allures  model  wherein  the  independent  subsystems  for  a 
series  system.  Some  of  the  papers  containing  these  derivations  were 
written  prior  to  the  publication  of  the  derivation  of  the  optimum  bounds 
[see  Takenaga  (49)  and  Kraemer (25) ] , 

Other  work  has  been  directed  at  providing  a  more  tractable  method 
of  calculating  confidence  bounds  than  that  of  El  Mawazlny's  generalization 
to  k  subsystems,  k>2,  of  the  Lentner-Buehler  bounds  which  apply  to  2  sub¬ 
systems  only  [see  El  Mawaz'iny  and  Buehler  (12),  Sarkar  (41)  and  Grubbs  (21)]. 
The  method  suggested  by  El  Mawaziny  and  Buehler  depends  upon  large-sample 
theory  and  the  others  use  the  fact  that  a  function  of  the  estimator  of  sub¬ 
system  mean- time- to- failure  has  a  chi-square  distribution.  The  method  of 
Sarkar  does  not  require  that  the  subsystems  be  independent  and  is  exact 
for  equal  numbers  of  failures  for  all  subsystems. 

Some  rather  limited  numerical  comparisons  have  been  made  of  some 
of  these  non-optimal  methods  for  obtaining  confidence  bounds  by,  for 
example,  Sarkar  (41)  and  Grubbs  (21).  Apparently  none  of  these  methods 
have,  until  this  time,  been  subjected  to  a  thorough  comparison  with  the 
Lentner-Buehler-El  Mawaziny  bounds,  which  must  be  calculated  iteratively 
from  an  expression  which  demands  extremely  complicated  calculations  when 
the  number  of  subsystems  is  more  than  two  or  three.  (Problems  involving 
loss  of  precision  and  use  of  excessive  amounts  of  computer  time  also  arise 
in  calculating  the  EL  Mawaziny  boundB  when  the  product  of  the  number  of 
Independent  subsystems  and  the  number  of  failures  for  any  given  subsystem 
is  more  than  about  30.) 
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Other  work  deal ins  with  the  derivation  of  confidence  bounds 

fur1  ovoinn  rol  4  qK  H  i  hi/  <M^or  rWn  ovnnfi  an  f*  -[  al  *>f  1  nT‘o»h  ^ma  mn^o  1  1  tlrl  tlHp 

a  Bayesian  approach  fit  a  parallel  system  with  a  single  failure  for  each 
subsystem  by  Springe..  and  Thompson  (47)  and  two  reports  by  Allen,  Carlson 
and  Hubach  (2)  and  Saunders  (42),  which  discuss  the  f lxed-teBt-time  model 
fcr  a  series  system. 


/or  the  case  in  which  only  pass-fail  data  are  collected  for  each 
subsystem  many  methods  involving  large-or  small-sample  approximations  or 
Bayesian  techniques  have  been  derived  for  obtaining  confidence  bounds  on, 
the  probability  of  successful  operation  of  an  Independent  series  system. 

Among  the  large-sample  methods  are  those  suggested  by  Madansky  (34) 

(based  on  the  asymptotic  chi-square  distribution  of  -2  log  likelihood 
ratio),  by  Myhre  and  Saunders  (37)  (which  gives  a  generalization  of  Madansky's 
method)  and  by  Rosenblatt  (40),  DeCicco  (11)  and  Thomas  (50)  (all  three  of 
which  are  based  on  the  asymptotic  normality  of  maximum-likelihood  estimators). 
The  methods  of  Rosenblatt  and  Madansky  are  discussed  and  compared  by  Myhre 
and  Saunders  (38) ,  who  demonstrate  that  the  likelihood  ratio  method  at¬ 
tains  its  asymptotic  properties  for  smaller  sample  sizes  than  the  methodj 
suggested  by  Rosenblatt  and  in  practical  situations  appears  to  yield  more 
accurate  bounds.  Madansky  (34),  however,  points  out  that  the  Rosenblatt! 
method  has  3lightly  higher  asymptotic  (Bahadur)  efficiency.  The  methods 
of  DeCicco  and  Thomas  use  Taylor-aeries  approximations  to  the  variance  of 
the  maximum  likelihood  estimator  of  the  syBtem  reliability  R  and  would  be 
expected  to  have  asymptotic  properties  like  those  of  the  Rosenblatt  method. 


Small-sample  approximate  confidence  bounds  on  R  for  an  independent 
series  system  and  binomial  data  have  been  derived  by  Nishime  (39)  ,  Garner 
and  Vail  (20),  Connor  and  Wells  (8),  Abraham  (1)  and  Lindstrom  and  Madden 
[see  Lloyd  and  Llpow  (33)].  The  first  three  of  these  approaches  use 
various  methods  of  combining  confidence  bounds  on  subsystem  reliability 
to  obtain  the  desired  bounds  on  sya  :em  reliability.  The  others  use  bi¬ 
nomial  or  PoiaBon  approximations  for  certain  statistics.  Some  of  these 
methods  are  sensitive  to  inequality  of  sample  sizes  for  subsystems.  Lower 
confidence  bounds  obtained  by  most  of  these  approximate  methods  have  been 
compared  by  the  use  of  three  sets  of  data  by  Schick  and  Prior  (44)  with 
three  different  sets  of  "exact"  bounds  obtained  using  results  of  Llpow 
[see  (31)  and  (32)],  based  on  Buehler's  theory  (6)  and  Poisson  approxima¬ 
tions.  The  data  apply  to  systems  composed  of  two  subsystems,  and  in  each 
of  the  three  cases  the  sample  sizes  are  equal.  Only  the  Lindstrom  and 
Madden  method  compares  favorably  with  what  appear  to  be  the  best  of  the 
Llpow  "exact"  bounds.  Since  there  is  some  question  about  the  standard 
used  to  judge  the  quality  of  the  approximate  methods,  howevtrt  and  since 
only  three  sets  of  data,  two  subsystems  and  equal  sample  sizes  have  been 
used  in  the  comparisons,  it  is  very  difficult  to  make  useful  general 
inferences  concerning  these  results. 


Another  method  investigated  numerically  by  Schick  and  Prior  (44)  is 
the  Bayesian  approach  wherein  reliability  for  each  subsystem  is  assumed  to 
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have  a  prior  distribution  which  is  uniform  over  the  unit  interval. 
Confidence  bounds  which  are  exact  in  the  Bayesian  sense  (under  the 
assumed  prior  distribution)  are  derived,  by  Zimmer,  Briepohl  and 
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use  a  Mellin  transform  technique  for  obtaining  in  closed  form  the 
distribution  of  system  reliability,  given  all  the  subsystem  data,  A 
Monte  Carlo  application  of  this  Bayesian  model  is  suggested  by  Mastran 
(3 b>  tor  a  system  which  is  logically  more  complex  than  a  series  system. 
In  the  numerical  comparisons  given  by  Schick  and  Prior  (44),  there  ap¬ 
pears  to  be  no  particular  agreement  between  the  sets  of  Bayesian  bounds 
calculated  on  the  basis  of  the  procedure  prescribed  by  Springer  and 
Thompson  and  by  Zimmer,  et  hi.  (Which  Incidentally  agree  to  two  or 
three  significant  figures,  as  one  might  expect)  and  the  three  sets  of 
"exact"  bounds  calculated.  In  particular,  the  Bayesian  lower  confidence 
intervals  on  R  are  all  larger  than  those  based  on  what  is  for  these 
three  sets  of  data  the  smallest  of  the  "exact"  intervals. 


One  would  expect  the  Bayesian  bounds  to  be  exact  in  a  classical 
sense  if  sample  sizes  for  all  subsystems  were  "sufficiently"  large.  This 
is  so  because  a  prior  density  of  the  assumed  type  will  have  less  effect 
upon  the  confidence  bound  as  the  sample  sizes  for  all  subsystems  increase. 
Whether  or  not  the  bound  Is  exact  in  a  classical  sense  has  not  been 
established.  Furthermore,  the  accuracy  of  this  bound  (see  footnote  1) 
has  not  been  investigated  for  small  sample  sizes.  It  Is  Interesting  to 
note  that  an  approximate  method,  described  by  Dalton  (10)  and  attributed 
to  TRW1 s  Florida  Operations,  yieldB  bounds  which  agree  to  within  3  in 
the  tnird  significant  figure  with  the  three  examples  calculated  in  (44) 
by  means  of  this  particular  Bayesian  approach.  The  TRW  method  has  the 
distinction  of  being  extremely  amenable  to  hand  calculation. 

Among  very  recently  derived  approximate  methods  for  obtaining 
confidence  bounds  on  the  probability  of  successful  operation  of  a  series 
system  are  (1)  those  derived  by  Woods  and  Borsting  (51)  (discussed  by 
Lieberman  (30)),  which  are  shown  by  Monte  Carlo  investigations  in  their 
paper  to  be  very  nearly  exact;  (2)  those  derived  by  J.  R.  Johnson  (23) 
based  on  the  exact  multi-variate  binomial  distribution  of  component 
test  data,  and  (3)  those  arising  from  a  Bayesian  approach  which  formally 
uses  subjective  judgment  concerning  prior  knowledge  by  J,  Bram  (5). 


The  Monte  Carlo  Confidence  Bound  Problem.  We  now  examine  the  problem 
of  obtaining  lower  confidence  bounds  on  the  reliability  of  a  logically 
complex  system  when  testing  will  be  performed  on  the  k  independent  sub¬ 
systems  only.  We  assume  that  an  equation  relating  true  subsystem  reli¬ 
abilities  to  true  system  reliability  ie  available,  say  by  meanB  of  computer 
programs  which  can  provide  such  information  [see  Levy  (28)  and  Mcknight, 
Modiest  and  Schmidt  (36)].  We  now,  in  lieu  of  an  appropriate  analytical 
method  of  obtaining  euch  bounds,  consider  the  possibility  of  the  use  of 
Monte  Carlo  techniques  aB  suggested  by  Burnett  and  Wales  (7),  BoBnikoff 
and  Klion  (4),  Costello,  Meisel  and  Letow  (9),  Levy  and  Moore  (29)  and 
Mastran  (35) . 
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Ac  first  glance  the  creation  of  a  Monte  Carlo  computer  program  for 
obtaining  the  bounds  seems  to  be  a  straightforward  problem  of  simulating 
the  distribution  of  system  reliability  for  a  given  set  of  failure  data  in 
.in  efficient,  manner,  it  soon  oecomes  apparent,  however,  that  there  are 
important  Bayesian  questions  implicit  in  the  problem.  That  is,  in  order 
to  generate  the  distribution  of  system  reliability  for  a  given  data  set, 
one  must  generate  for  each  subsystem  what  is  essentially  the  posterior 
distribution  of  subsystem  reliability,  given  the  subsystem  life-test 
failure  data.  Hence,  some  prior  distribution  or  something  equivalent 
to  such  a  prior  distribution  for  subsystem  reliability  must  be  implicitly 
or  explicitly  assumed.  In  other  words,  in  carrying  out  the  Monte  Carlo 
approach  outlined  by  the  authors  mentioned  above,  one  uses  the  denairy 
of  some  appropriate  function  of  the  data  and  implicitly  or  othervlae 
combines  this  information  with  a  prior  density  of  subsystem  reliability 
bv  means  of  Bayes'  Theorem,  P(A  |B)  -  P(B |A. )P(A. )/  ][  P(B|A  )P(A.), 

all  j  3  ,  3 

to  obtain  the  posterior  density  function  of  subsystem  reliability,  given 
the  data.  In  agreement  with  the  classical  analytical  method  derived  in 
(48),  the  Monte  Carlo  procedures  described  in  (4),  (7),  (9),  (29),  and 
(35),  in  some  cases  directly  suggest  and  in  others  tacitly  Imply  a  prior 
distribution  for  subsystem  reliability  which  is  the  appropriate  prior 
leading  to  the  classical  optimum  bounds  when  the  system  consists  of  one 
subsystem  only.  One  may  then  inquire  as  to  whether  such  an  assumption  is 
appropriate  when  the  system  consists  of  more  than  one  subsystem. 

Springer  and  Thompson  (47)  analytically  derive  their  exact  Bayesian 

confidence  bounds  on  R(t  )  for  an  exponential-failure-time  model,  wherein 

m 

one  failure  is  allowed  for  each  Independent  subsystem  of  a  parallel  system, 
using  an  alternative  a  priori  assumption.  They  eeeume  a  uniform  prior 
distribution  on  subsystem  reliability  over  the  unit  Interval,  which  leads 
to  the  classical  optimum  bounds  on  successful  system  operation  for  the 
pass-fall  model  when  the  system  consists  of  a  single  subsystem.  Springer 
and  Thompson  reason  that  a  flat  prior  for  subsystem  reliability  is  in 
keeping  with  the  Intent  of  Bayes'  Theorem  when  no  prior  information  la 
known.  They  point  out  that  the  prior  density  p(Rj)  for  the  Jth  subsystem 

reliability  yielding  the  classical  optimum  bounds  for  a  system  containing 
a  single  subsystem  and  an  exponential  fixed-failures  model,  p(Rj)  ■ 

R^1  [ln(l/Rj)]_1  or  equivalently,  q(Xj)  ■  Xj1.  where  ■  Rj(tm)  " 

(- X  t  )  and  0<R. 1 ,  j-l,2,...,k,  is  "improper"  in  that  the  area  under  the 
J  m  ~  J~ 

frequency  curve  cannot  be  made  equal  to  unity.  Mastran  (35)  suggests  for 
pass-fail  data  that  prior  densities  for  subsystems  which  lead  to  a  uniform 
prior  density  for  system  reliability  might  be  appropriate.  In  other  words 
all  the  suggested  prior  distributions  art  derived  from  the  concept  of 
oprimality  for  one  subsystem  for  some  model,  even  Chough  the  model  may 
have  little  relationship  to  the  one  of  Interest. 
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In  the  following,  a  description  is  given  of  results  of  n  study  (by 
members  of  a  group  at  North  American  Rockwell  Corporation)  to  determine 
optimum  prior  assumptions  to  be  used  in  generating  Monte  Carlo  confidence 
oounus  on  tne  reliability  ot  a  logically  complex  system.  The  investigation 
was  conducted  principally  by  K.  W.  Fertig  of  Rocketdyne  Division  and  the 
present  author.  Mr.  Ferrlg  wrote  all  computer  programs  needed  for  the 
investigation,  except  for  one  routine  linking  the  Monte  Carlo  program  to 
the  reliability  equation  for  the  complex  system.  He  also  provided  [see 
(17)]  the  important  analytical  derivation  of  the  necessary  form  for  a 
special  restricted  model  of  an  optimum  prior  density  function  independent 
of  the  data  and  proved  that  no  such  prior  density  exists.  Jerome  Spanier 
of  the  North  American  Rockwell  Science  Center  provided  consultation  on 
problems  related  to  the  Monto  Carlo  computer  program.  Shirley  Stoneberger 
of  the  Los  Angeles  Division  wrote  the  subroutine  which  makes  use  of  the 
reliability  equation  generated  from  engineering  flow  chart  information  by 
the  SCOPE  (28)  or  the  ARMM  (36)  program  for  a  logically  complex  system. 


RESULTS  OF  INVESTIGATION 


Computer  Programs  Written  and  Utilized.  An  optimum  standard  against 
which  to  judge  suggested  prior  distributions  provides  a  means  of  attacking 
the  Monte  Carlo  problem.  Therefore,  the  model  was  first  restricted  to  a 
series  system  wherein  the  Jth  independent  subsystem  has  exponentially 
distributed  failure  time  Tj  with  Prob  [Tj>tn  1  ■  ■  Rj(tB>  *  exp  (-1^^) 

and  tij  prototypes  of  the  Jth  subsystem,  j-l;2,...,k,  are  tested  until  r^ 

failures  occur.  If  one  can  determine  an  appropriate  prior  distribution 
for  this  model,  then  it  should  also  be  possible  to  make  useful  inferences 
concerning  the  fixed-failure-time  series-system  model  and  to  datarmina  a 
method  of  using  prior  information  for  more  complex  systems. 


A  computer  program  was  coded  In  Fortran  H  for  the  IBM  S/360  system 
for  calculating  for  this  veatrictad  model  the  optimum  classical  confidence 
bounds  of  Lentner,  Buehler  and  El  Mavaziny  discussed  In  the  introduction  of 
this  paper.  The  bounds  are  based  on  the  conditional  distribution  of  W  ■  Z 

TJ 


1* 


given  Zj-Zj  -  u2,...,  Z^Z^ 


u,  ,  where  Z,  - 
k*  j 


l 

1-1 


Ti,j  +  (nj’rJ)Tr3.J' 


with  T  ,  an  observable  failure  time  of  the  ith  prototype  of  the  Jth 

component,  and  where  the  subscript  1  is  arbitrarily  assigned.  Then,  whan 
Uj  is  less  than  zero  for  j-2,3...,k,  the  optimum  claselcal  (1-oO-leval 

lower  confidence  bound  Rn(a)  on  R(t  )  -  oxp(-0t  ) ,  (where  0  -  )  1.)  is 

o  in  m  J 


j-i 

obtained  by  finding  the  solution  0g(a)  the  following  equation  and  then 
calculating  Rfl  -  exp  [-0g(a)  tffl]  ,  with  0g>O, 
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k 


H  (w|u;$)  •  A  ^  (u j  if)  Z  £ 

J  4 

x2  ‘3 


Z  T! 

J  A  — 'I 

Ak  J~‘ 


(*i\  C-u)V*j' 
P/  J 


V1 


$w 


(a^Jj  ij+1) 


1-a, 


where 


A<u; ♦) 


-1- 

* 


*k  k 

I  (a  +  £  i  ) 

*k  J-2  j 


(1) 


(5) 


and  where 


Vi  *  1  ‘j+1> 


e~*y  dy 


A  similar  expreasion  is  used  If  any  u . ,  J-2,3,...,k,  la  greater  than 
zero,  and  the  solution  la  obtained  by^ joint  application  of  Newton- 
Raphson  Iterative  procedures,  the  method  of  false  position  and  bisection 
techniques.  Then  a  computer  program  for  generating  Monte  Carlo  confidence 
bounds  was  coded  and  combined  with  that  for  obtaining  the  Lentner-Buehler- 
E1  Mawazlny  confidence  bounds.  A  listing  and  flow  chart  of  the  combined 
computer  program  are  available  [see  Fertlg  (16)  and  (18)]. 

The  Monte  Carlo  program  calculates  the  confidence  bounds  on  the  basis 
of  a  specified  prior  density  for  subsystem  reliability  which  is  a  member 
of  the  "conjugate"  family  of  prior  densities.  That  is,  the  prior  density 
yields  a  posterior  density  of  subsystem  reliability,  given  the  subsystem 
data,  of  the  same  general  form  (belonging  to  the  same  family  of  density 
functions)  as  the  prior  density.  The  prior  density  function  p,(R.)  used 
for  the  Jth  subsystem  reliability  was,  therefore,  J  3 
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f.  ^ro.1+1 


r fR.)  -  ZailSl  R,Cuj  find/R,)] 


uj 


/n\ 
\  / 


”  r<roi+1> 


VoJ  ,‘1'  J  '  1,2 . *•  vit’'  ‘oj  “a  ro< 


subjectively  chosen.  This  yields  a  posterior  density  f(Pj(B  ;r^)  for  Rj 

of  the  form,  +  +^rj+r0j+l 

f(F.|8  ;r  )  «  — — — -  R  6013  [ln(l/R  )lrJ+roJ 

J  J  i  r(r.+r  ,+l)  i  J 


(3) 


J  oj 


where  the  random 


variable  Z,/t_  *  (f1  T.  +(n -r.)T  .)/t 

J  “  4,1  U  J  J  r,>J  m 


is  equal  to  *  /t  *6  ,J«1,2, . . . ,k ,  for  the  observed  set  of  data. 

J  m  J 


If  and  each  have  the  value  -1,  then  the  prior  density  for  Rj 

corresponds  to  the  "improper"  prior  which  is  used  by  (48),  (14),  (7) ; 
and  (29)  and  which  gives  the  optimal  classical  boundB  for  a  system 
consisting  of  a  single  subsystem  [see  Epstein  and  Sobel  (14)],  that  is, 


p(Rj)  <•  Rj1-  [ln(l/Rj)]  1 ,  J«l,2 . k.  (It  is. true,,  therefore,  that  even 

though  the  prior  density  corresponding  to  Boj  ■  rQj  -  -1  is  "improper," 

the  corresponding  posterior  density  is  proper.)  If  and  r^  are  both 

equal  to  zero,  J-1,2 , . . .  ,k,  then  each  subsystem  prior  density  function 
for  subsystem  reliability  is  uniform  over  the  interval  from  0  to  1,  as 
suggested  by  Springer  and  Thompson  (47)  for  their  special  case  of  a 
parallel-system  model  mentioned  earlier. 


For  generating  the  posterior  distribution  of  R,  using  the  expression 
(3) ,  a  given  aet  of  data  and  specified  values  for  63  ^  and  r^ ,  a  random 

number  Oj  is  geperated  for  the  value  of  the  integral  Yj  a  I Bj ;r ^ ) 

given  by  the  expression  (3)  from  R  ,  to  1,  The  integration  is  performed 

y  »j 

by  an  evaluation  of  the  incomplete  gamma  function  and  the  value  of  R 

Y » j 


determined  iteratively.  The  Newton-Raphaon  method  of  iteration  in 
conjunction  with  the  method  of  false  position  is  used.  Because  this 
procedure  is  quite  expensive  in  terms  of  computer  time,  the  computer 
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program  was  written  to  calculate  for  a  given  set  of  subsystem  data  a 
table  of  100  values  of  corresponding  to  equally  spaced  values  of 

y(Rj  |  ;r^ ) .  The  computer  then  samples  from  and  interpolates  cublcally 

in  this  table  for  a^<Pj<a^  where  a^  and  a ^  are  functions  of  the  data. 

For  ~  and  p  ^a0,  a  different  table  is  sampled.  In  generating  values 

for  this  table,  > (R ^  • r j )  calculated  from  a  specified  value  of 

k  ,  so  that  no  iteration  is  necessary but  the  values  of  y  are  not 
4  t  J  J 

equally  spaced  (making  interpolation  more  difficult).  The  second  table, 
which  contains  values  of  much  closer  together  than  the  one  used  for 

non-extreme  values  of  R  ,  is  necessary  because  of  the  steepness  of 

Y » J 

the  curve  reLating  y.  and  R  for  values  of  R  .  close  to  0  or  1 . 

J  Y» J  Y i j 

.  The  first  investigation  made  by  means  of  the  computer  was  of  the 
two  familiar  prior  distributions  corresponding  to  and  both  equal 

to  rero  and  both  equal  to  -1,  J»l,2,...,k.  The  Bayesian  approach  cor¬ 


responding  to  & 


-  r  -  -1,  incidentally,  is  sometime^  called  the 


fiducial  model  since  ^he  posterior  distribution  of  R^ ,  J“l,2,...,k,  can 

be  thought  of  as  chra/inable  from  the  distribution  of  a  function  of  the 
data  for  the  jth  subsystem,  as  detailed  in  (25).  The  preliminary  phases 
o!  this  Investigation  made  use  bf  the  Monte  Carlo  program,  but  the  results 
given  below  were  obtained  using  instead  a  computer  program  which  utilizes 
a  Mellin  transform  t/echnique  [see  Springer  and  Thompson  (46)]  to  calculate 
the  posterior  distribution  of  R(t  )  from  the  posterior  distributions  of 

the  R^'s.  This  Mellin  transform  program  was  originally  written  to 

calculate  the  variance  of  the  Monte  Carlo  confidence  bound  and  is  applicable 
to  a  series  system  when  the  posterior  density  of  Rj  has  the  form  given  by 

the  expression  (3)  with  r^  an  integer.  The  Mellin  transform  computer 

program  is  faster  than  the  Monte  Carlo  program  and  gives  better  precision, 
but  in  its  present  form  cannot  be  uBed  if  roj  is  other  than  an  integer. 

Study  of  Suggested  Prior  Densities.  For  each  combination  of  input, 


involving  from  three  to  twenty-five  components  having  X's  in  various 
proportions,  numbers  of  failures  ranging  from  1  to  10  and  three  or  four 
different  values  of  a  ranging  from  .05  to  .50,  data  were  generated  and  a 
comparison  was  made  of  the  two  Bayesian  bounds  with  the  optimum  classical 
bound.  In  each  case  (of  a  total  of  156  casee),  the  Bayesian  bound  based 

on  6  ■  r  ,  ■  -1  Is  smaller  than  the  corresponding  classical  bound 

oj 

obtained.  It,  therefore,  appears  that  though  exact  in  a  Bayesian  sense 
(under  the  assumed  prior  distribution  for  R^ , j-1,2 , . . >k) ,  the  bounds  based 

on  such  a  prior  assumption  are  conservative  In  the  classical  sens#. 
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When  the  optimum  bound  is  standardized  at  .800  by  adjusting  the 

mi «« i nri  Mmp  r  .  t-he  fiducial  hound  ranees  from  .538  to  .793.  When  the 
m ' 

optimum  bound  is  equal  to  .368,  the  fiducial  bound  ranges  from  .062  to 
.  354. 


El  Mawaziny  and  Buehler  (13)  show  that  their  large-sample  approximation 
of  the  optimal  bound,  a  bound  obtained  by  the  Rosenblatt  method  (40)  and 
tne  ilducial  bound  will  approach  the  optimal  bound  as  numbers  of  failures 
for  all  subsystems  become  large.  For  three  samples  having  ten  failures 
for  each  of  three  identical  components,  the  fiducial  bounds  were  of  the 
order  .787  and  .341  for  optimal  bounds  6f  .800  and  .378,  respectively, 
with  deviation  between  any  two  corresponding  fiducial  bounds  less  than 
three  in  the  third  decimal  place. 

Analytical  results  described  later  indicate  that  the  fiducial  bounds 
for  a  fixed  number  of  failures  per  subsystem  will  agree  less  well  as  the 
number  of  subsystems  increases  and  the  subsystems  become  more  variable 
with  respect  to  failure  rate.  Unfortunately,  because  of  the  computer-time 
factor  and  considerations  of  precision,  it  is  impossible  at  present  to 
compare  bounds  for  systems  containing  aa  many  as  ten  subsystems  when  as 
many  as  ten  failures  occur  for  more  than  one  or  two  of  these  subsystems. 

In  any  case,  the  large-sample  methods  cannot  be  expected  to  giye  bounds 
agreeing  well  with  the  optimal  bounds  when  aome  of  the  subsystems  have 
been  subjected  to  few  tests.  Furthermore, . it  1b  impossible  on  the  basis 
of  these  results  to  say  whether  bounds  baaed  on  this  specified  prior  might 
be  conservative,  liberal,  or  exact  for  a  particular  logically  complex  system. 

The  uniform  prior  distribution  for  subsystem  reliability  gives  bounds 
even  lower  than  those  based  on  the  fiducial  method  except  in  24  caaes  (out 
of  150)  ^n  which  all  three  bounds  have  values  fairly  close  to  zero;  In 
these  24  cases  they  are  higher  than  the  optimum  classical  bounds.  It  ap¬ 
pears  that  the  distribution  of  the  bounds  baBed  on  the  uniform  prior  may 
be  less  disperse  than  those  of  the  optimal  bounds,  but  these  bounds  seem 
to  be  even  more  conservative  than  the  "fiducial"  bounds  given  by  6^  ■  rQj 

•  -1  for  true  reliabilities  of  a  reasonable  size  and  a’s  of  interest. 

For  systems  with  low  reliabilities,  bounds  obtained  using  a  uniform  prior 
density  for  subsystem  reliability  should  be  liberal  rather  than  conservative 
when  the  confidence  level  la  sufficiently  low,  but  not  exact  in  general. 

This  inconsistent  behavior  may  be  due  to  the  fact  that  a  uniform  prior 
density  for  R^  Implies  a  prior  density  for  1^  (the  failure  rate  for  the 

Jth  subsystem)  of  the  form  q(A.)  ■  t  exp(-A,  t  ),  j*l,2,...,k,  or, 

j  m  j  m 

strangely,  one  which  Is  a  function  of  t^,  the  specified  mission  time. 

The  result  for  “  r^  -  -1  is  keeping  with  the  analysis  and 

numerical  results  of  Saunders  (42) ,  who  studies  a  fixed-teat-time 
exponential  serteB-system  model  and  the  Bayesian  approach  suggested  in 
(2).  This  Bayesian  approach  uses  a  prior  density  for  the  flxed-test-time 
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codel  equivalent  to  the  so-called  fiducial  method.  Saunders  (42) 
nolnts  i-ut  that  in  using  such  a  Bayesian  model  for  an  exponential 
series  system  (and  his  argument  applies  to  any  true  Bayesian  model, 
that  is,  one  based  on  a  prior  assumption  which  does  not  involve  in¬ 
formation  concerning  the  number  of  components  in  the  system) ,  one  can 
obtain  different  confidence  bounds  depending  upon  what  one  chooses  to 
all  a  subsystem.  Saunders  points  out,  too,  that  such  inferences  apply 
vo  more  logically  complex  systems  which  are  highly  reliable,  since  one 
can  approximate  an  extremely  reliable  coherent  syetem  [see  Birnbaum, 

Lsary  and  Saunders  (3)  for  a  definition  of  a  coherent  system]  by  a 
series-system  model,  as  indicated  by  Esary,  Proschan  and  Walkup  (15). 

The  Search  for  Optimum  Prior  Assumptions.  Initially,  it  has  been 
planned  that  a  trial,  and  error  procedure  would  be  used  in  attempting  to 
determine  appropriate  prior  assumptions  for  our  "series-system  model. 
Saunders'  argument  might  lead  one  to  consider  trying  prior  assumptions 
which  are  not  truly  Bayesian  in  that  they  are  dependent  upon  the  con¬ 
figuration  (or  number  of  components  in  the  system).  At  this  point 
in  the  study,  however,  an  analytical  result  was  derived,  modifying  the 
subsequent  approach.  The  details  of  the  analysis  are  given  by  Fertig  (17) 
and  are  summarized  below. 

First,  the  form  of  a  prior  density  function,  or  generalization  of 
such  a  function ,  for  Rj ,  j-l,2,...,k,  corresponding  to  the  optimum 

classical  bounds  for  our  system  model  was  determined.  This  was  accomplished 
by  setting  the  Laplace  transform  of  (w|u}$)  equal  to  the  Laplace  trans¬ 
form  of  the  posterior  density  of  $  ■'  l  X.,  obtained  under  a  general 

J-l  3 

prior  assumption  (not  restricted  to  conjugate  priors)  for  the  special  case 
u2  •  u~  -  . . .  ■  «  0.  If  the  prior  assumptions  which  yield  the  optimum 

bounds  are  independent  of  the  data,  true 'Bayesian  prlora,  for  example, 
then  an  assumption  concerning  the  value  of  the  u’a  will  have  no  effect 
on  the  result.  The  fact  that  the  optimum  classical  bounds  arc  Invariant 
under  permutations  of  the  subsystems  was  used  to  obtain  the  Improper 
"prior  density"  for  the  ^th  subsystem  yielding  these  bounds.  It  is 

p(Rj)  -  R"1  [ln(l/Rj)]"(2“1/k)  ,  3-1,2 . k. 

We  note  that  this  Improper  density  depends  upon  k,  the  number  of  subsystems 
In  the  series  system,  and  for  k-1  does  yield  the  optimum  classical  bound. 

The  Monte  Carlo  program  was  then  used  to  test  whether  this  prior 
assumption  (which  we  may  think  of  as  a  weighting  function  sines  it  does 
not  correspond  to  a  strict  Bayesian  prior  density)  would  yield  the  optimum 
ooauds  for  variations  in  the  data.  For  the  case  where  all  the  u’s  equal 
zero  (z^  -  z,  -  ...  -  z^) ,  eight  values  of  the  Monte  Carlo  bounds  based 
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on  5,000  replications  agree  with  the  optimum  bounds  to  within  three  In 
the  third  decimal  place.  Data  were  randomly  generated  for  five  subsystems 
using  the  fact  that  TL.h  is  distributed  as  chi-square  with  2  degrees 

J  r1 
ot  freedom  [see  Epstein  and  Sobel  (14)],  where  *  3,  r2  -  4,  r^  ■  4, 

r4  =  2,  r5  -  2,  *  1/12,  X2  -  1/13,  ^  -  1/15,  *4  -  1/10,  ^  -  1/11. 

The  mission  time  was  taken  as  1.0.  For  ten  such  data  sets,  the  Monte  Carlo 
bounds  are  uniformly  ldrger  than  the  classical  confidence  bounds  with 
deviations  ranging  from  1  to  6  in  the  second  decimal  place. 

The  confidence  bound  obtained  from  any  set  of  data  by  El  Mawaziny's 
formulia  given  by  Eq.  (1)  is  the  unique  optimum  (uniformly  most  accurate 
unbiased)  confidence  bound  for  this  exponential-failure-number  series- 
system  model.  This  is  proved  in  the  Appendix  of  Fertlg's  paper.  Thus, 
any  optimum  bound  is  equivalent  to  the  bound  defined  by  Eq.  (1)  and  must 
give  the  same  result  for  any  given  set  of  data.  Since  the  error  in  the 
Monte  Carlo  procedure  is  very  small  compared  with  the  deviations  obtained 
for  the  u's  nbt  all  equal  to  zero,  the  empirical  evidence  indicates  that 
the  prior  assumptions  which  yield  the  optimum  classical  bounds  do  depend 
upon  the  data. 

Fortunately,  a  means  of  proving  this  result  analytically  then 
presented  itself  [see  Fertig  (17)  for  details].  The  Laplace  transform 
of  H(w|u;<|>)  is  a  horrendous  expression  which  gives  no  apparent  clue  as 
to  how  it  might  be  factored  and  assigned  to  the  various  subsystems.  The 
problem  was  made  tractable  earlier  by  letting  all  the  z's  be  equal. 

Another  method  of  simplifying  the  expression  was  found  to  be  to 
assume  r^  -  t2  ■  *  1.  If  this  is  done,  then  it  is  possible  to 

demonstrate  that  the  "prior  density"  for  the  jth  subsystem  yielding  the 
optimum  bound  for  r^  *  r^  ■  ...  ■  r^  ■  ,1  cannot  have  the  form  p(Rj)  ■ 

“1  '  -(2-1/k) 

Rj  [ln(l/R)]  unless  z^'-  z2  ■  ...  ■  z^,  as  assumed  in  the 

earlier  case.  Hence,  as  indicated  by  the  numerical  evidence,  one  must 
incorporate  present  data  into  the  "prior  assumptions"  or  more  properly 
the  weighting  functions,  for  obtaining  optimum  confidence  bounds  for  the 
exponential  fixed-failure-number  series-  system  modej.. 

Now  that  we  have  established  what  is  not  fruitful  for  obtaining  the 
optimal  bounds,  one  may  properly  inquire  as  to  the  next  step  in  the 
investigation  with  respect  to  confidence  for  a  complex  system.  Two 
approaches  present  themselves.  The  first  is  to  consider  each  series 
system  which  is  a  part  of  the  complex  system  as  a  single  subsystem  of 
the  tor.al  system.  If  the  fiducial  approach  were  to  be  used,  one  could 
obtain  an  estimate  R  of  true  reliability,  given  the  failure  data,  for 

£  I  *• 

the  _£th  subsystem  consisting  of  the  independent  subsystems  making  up 
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this  series-system  subsystem  by  substituting  a  random  number  .pi  f°f  . 

1  t  —  L'm  /IN  ThnM  ►  U  frtl  <  H  nniiaf  4  nn  frtil  1  H  Ka  IIAaH  t*ft  Ahfill  fl 

4  -  *  •  •■-**  •  \*-/»  **“••**  «•  v  —  “  1 - - -  -- 

an  estimate  of  total  system  reliability  given  the  failure  data.  This 
method  will  probabLy  not  yield  confidence  bounds  that  are  exact  in  the 
classical  sense,  though  of  course  they  are  exact  in  a  Bayesian  sense. 

In  lieu  of  the  fiducial  approach  one  could  somehow  modify  the  value 

obtained  for  R  .  in  an  attempt  to  obtain  bounds  exact  in  the  classical 

0  ,  k. 

sense.  Clues  as  to  how  this  might  be  accomplished  may  possibly  be  ob¬ 
tained  by  Investigating  a  simple  parallel  system  (again  an  exponentlal- 
failure-time  model  with  fixed  number  of  failures)  and  methods  of  obtaining 
confidence  bounds  exact  in  the  classical  sense  for  the  simpler  model. 


A  considerable  amount  of  computer  time  will  be  required  with  this 
approach  when  the  product  of  the  number  of  subsystems  in  any  serlem 
system  and  the  number  of  failures  for  any  subsystem  in  that  series 
system  becomes  large,  l'er.ce,  one  might  in  such  cases,  abandon  the  idea 
of  considering  the  scries  system  as  a  single  subsystem.  Instead  one 
might  consider  a  method  of  approximating  the  optimum  confidence  bound 
for  a  series  system  by  using  series  system  d^ta  in  the  prior  assumptions 
(or  weighting  functions)  for  the  subsystems  of  'the  series  system  that 
yield  the  posterior  distribution  of  series  system  reliability.  For  the 
case  T  i  "  r2  *  '  • '  rit  "  one  P°9sibl-e  factoring  of  the  Laplace  transform 

gives 

p  ^  (Rj )  *  Rj^  **jr^(l)  [ln(l/Rj)J  ^  f  j*l*2p. . .  ,k^,  (4) 

where  8^  is  the  smallest  of  STj/t^  z2/t m . "Hi/V  Thls  function  or 

some  modification  of  this  function  involving  the  true  values  for  r^, 

....  rfc  could  be  tried.  If  the  expression  (4)  is  used  without  modifica¬ 
tion  ,  then  the  posterior  distribution  of  series  system  reliability,  given 
the  failure  data,  depends  only  upon  It  can  be  shown  that,  in  fact, 

the  fiducial  distribution  of  series  system  reliability  for  this  model 
depends  only  upon  8^  if  and  only  if  r^  ■  rj  *  *1. 


The  method  of  Kraemer  (24)  depends  solely  upon  B^  and  hence  would 

be  expected; to  give  poor  results  for  large  numbers  of  failures  per 
component.  This  is  born  out  by  the  comparisons  made  by  Sarkar  (42)  and 
Grubbs  (21). 


The  bounds  derived  by  Grubbs  (21)  are  approximations  to  the  fiducial 
bounds,  and  for  the  bounds  compared  during  this  study,  the  approximation 
appears  to  be  excellent.  The  Grubbs  method  is  based  on  the  fact  that 


2r 


J 


Y; 


Is  distributed  as  chi-square  with  2r^  degrees  of  freedom, 
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X 


j»l  ,2 , . . . ,k,  so  that 


Xj  can  be  thought  of  as  distributed  as  a 


weighted  sum  of  chi-square  variates.  The  weights  used  by  Gtubun,  namely, 
Xj/2Tj,  j«l,2,...,k,  are  appropriate  for  obtaining  the  fiducial  bounds. 

One  could  obtain,  instead,  an  approximation  to  the  optimal  bounds  by 
adjusting  the  weights  appropriately,  obtaining  clues  from  the  expression 
(4)  above.  In  this  way,  one  can  test  approximations  to  the  optimum  prior 
assumptions  and  avoid  the  time-consuming  Monte  Carlo  calculations.  If 
successful  in  closely  approximating  the  optimum  bounds  by  the  proper 
modification  of  the  Grubbs  method,  one  can  use  this  approximation  in 
place  of  the  Lentner-Buehler-El  Mawaziny  bounds  in  considering  series 
systems  within  a  complex  system  as  single  subsystems. 

The  investigation  is  being  continued  along  these  lines. 
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COMPARATIVE  ANALYSIS  OF  THE  LCSS-ETG-3  PERFORMANCE 
CAPABILITY  USING  STATISTICAL  PROBABILITIES 

Andrew  H.  Jenkins 
U.  S.  Army  Missile  Command 
R.cdst:c>i'ic  Aj-'seual  i  Alabama 

ABSTRACT 

A  test  plan  is  formulated  and  executed  to  obtain  a  random  sample  of 
measurements  on  the  U.S.  Army  Missile  Command's  Land  Combat  Support 
System  Electronic  Test  Group  equipment. 

Analyses  of  the  data  establish  sample  estimates  of  bias,  accuracy,  and 
stimulus  setting  errors  and  the  standard  deviation  of  measurement  on  14  com¬ 
binations  of  parameters  and  scales  (e.g,,  dc  voltage,  10-volt  scale).  The 
analyses  pose  hypotheses  about  the  statistics  and  test  these  hypotheses  against 
appropriate  frequency  distributions.  They  include  the  principle  of  analysis  of 
variance,  which  makes  use  of  bias  error,  accuracy  error,  stimulus  setting 
error,  and  sample  variance.  These  four' parameters  are  used  as  response 
variables  to  establish  the  effects  of  the  main  factors  of  test  durations,  time 
delays,  and  machines  and  combinations  of  the  main  factors  (i.e.,*  Interactions) 
*  oil,  the  computed  response  statistics  for  each  of  the  14  parameters  and  scales 
considered,,  .  -■ 

\  The  overall  estimates  of  the  precision  (standard  error  of  measurement) 

for  each  parameter  and  scale  are  related  to  actual  weapon  system  tolerances 
to  obtain  probability  estimates  of  the  risk  of  passing  a  bad  unit  |  "undetected 
defect")  or  holding  a  good  unit  ( "false  alarm")  in  a  Single  test  in  a  checkout 
procedure,  (  single  checkout  probabilities  are  related  to  multiple  sequential 
checkout  probabilities,  ^ 
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Section  I.  INTRODUCTION 


The  purpose  of  the  Land  Combat  Support  SyBtem  (LCSS)  electronic 
test  group  tETG)  equipment  is  to  provide  maintenance  support  and  check 
operational  readiness  of  major  modules,  assemblies,  and  subassemblies  of  the 
Shillelagh,  Lance,  and  TOW  missile  systems.  The  primary  requirement  is  for 
direct  and  general  support  missions,  A  detailed  description  of  the  LCSS-EtG 
can  be  found  in  a  previous  report  [  1) .  The  ETG  is  designed  to  automate  the 
testing  of  critical  components  of  t.he  missile  systems  to  achieve: 

a)  Rapid  evaluation  of  tho  operational  status  of  the  unit  under  test 

' ( UUT i . 

b)  Rapid  fault  isolation  of  a  defective  UUT. 

c  >  Automated  decision  iriakirtg  as  to  operational  status  by  comparison 
^  of  measured  values  with  prescribed  standards. 

d(  A  standardized  automated  teat  capability  for  several  weapon  systems. 

Automation  of  the  ETG  equipment  requires  the  preparation  of  a 
programmed  test  sequence.  The  test  program  instructs  the  operator  on  the 
required  manual  operations  for  the  checkout  such  as  external  connections  to 
"make"  and  "break."  The  program  includes  all  necessary  tests  for  function^ 
checkout  of  the  UUT's  as  prescribed  by  the  weapon  system  design  engineers. 
Typical  tests  are  to  measure  stimuli  and  responses  of  such  parameters  as  ac  and 
dc  voltage,  resistance,  optical  alignments  frequency,  phase,  and  time,  and  to 
compare  these  measurements  with  prescribed  values.  The  required  values  of 
such  parameters  along  with  acceptable  tolerances  (deviations)  are  prescribed 
in  the  test  program.  The  test  equipment  makes  the  measurement  and  compares 
it  with  the  specified  value  and  decides  on  a  "go/no-go"  basis  as  to  a  fault 
determination. 

In  the  testing  of  missile  components  on  a  go/ no-go  basis  there  are  a 
combination  of  conditions  which  may  exist,  X  unit  may  be  good  and  check  goed 
resulting  in  a  go  decision.  The  unit  may  be  good  and  check  bad  resulting  in  a 
no-g.  decision.  On  the  other  hand,  a  unit  may  be  bad  and  check  good  resulting 
in  a  go  decision,  or  it  may  be  bad  and  check  bad  resulting  in  a  no-go  decision, 
Thete  are  certain  probabilities  associated  with  these  combinations  of  actual 
component  condition  and  checkout  results.  These  are  shown  in  Table  I,  The 
pun  is  the  probability  that  a  unit  checks  good  when  in  fact  it  is  bad.  The  P(3) 

Is  the  probability  that  a  unit  checks  bad  when  in  fact  it  Is  good.  Some  authors 
refer  to  these  probabilities  as  an  "undetected  defect"  and  a  "false  alarm," 
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TABLE  I.  CHECkOUT  PROBABILITY  VERSUS  UNIT  CONDITIONS 
AND  TEST  DECISIONS 


Unit  Condition 
and  Checkout 

Decision 

Go 

No-Go 

Bad 

Checks  good  (undetected  defect) 

p(O) 

NA 

Checks  bud 

NA 

(1  -  p  </3)  ) 

Good 

Checks  good ; 

1 1  -  P(«>  1 

NA 

Checks  bad  (false  alarm) 

NA 

P(0) 

respectively."  It  can  be  seen  that  the  p(a)  is  related  to  a  go  decision  and  the 
p(/3)  is  related  to  a  no-go  decision  based  on  the  test  results.  A  unit  that  is 
good  and  checks  good  will  n6t  result  In  a  no-go  decision.  Similarly,  a  unit  that 
is  bad  and  checks  bad  will  not  result  in  a  go  decision.  These  combinations  are 
not  applicable  and  ate  shown  as  NA  in  Table  I.  In  a  go  decision  situation  there 
is  a  probability  that  a  bad  unit  has  checked  good;  i.e'. ,  there  is  ap(a)  chance 
that  a  defect  exists  and  it  is  undetected  by  the  test  equipment  which  Is  an 
"undetected  defect. "  In  k  no-go  decision  situation  there  is. a  probability  that  a 
good  unit  has  checked  bad;  i.e..  there  is  a  p  (0)  chance  thal,  the  test  equipment 
has  falsely  indicated  a  defect  that  does  not  exist  which  is  a  "false  alarm.  " 
Therefore,  it  is  seen  that  the  p(a)  is  the  probability  of  simultaneously  getting 
a  measured  value  within  the  specification  limits  (or  the  decision  limits)  and 
an  actual  value  outside  the  specification  limits.  The  p(0)  is  the  probability  of 
simultaneously  getting  a  measured  value  outside  the  specification  limits  (or 
decision  limits)  and  an  actual  value  inside  the  specification  limits. 

In  other  words,  given  a  go  decision,  the  probability  that  it  is  wrong  (bad 
checks  good)  is  p(O)  and  the  probability  that  it  1b  right  (good  checks  good)  Is 
( 1  -  p (<*)].  Given  a  no-go  decision,  the  probability  that  it  is  wrong  is  p((?) 
and  the  probability  that  it  is  right  is  ( 1  -  p  (0)  J , 

The  p(cv)  and  p(0)  set  for  the  test  equipment  should  be  realistically 
determined  in  light  of  the  weapon  mission  and  test  equipment  environment.  If 
the  probability  p(a)  is  set  too  high,  an  excessive  number  of  bad  units  going  to 
the  troops  will  result. 
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On  the  other  hand,  if  p  ((3)  is  too  high,  an  excessive  amount  of  time  is 
spent  in  checking  for  a  defect  that  does  not  exist.  Therefore  experience, 
knowledge  of  military  tactics  and  good  judgment  should  govern  the  compromises 
between  logistics,  field  troop  effectiveness,  troop  operational  conditions, 
military  objectives,  etc. ,  to  determine  the  levels  of  P(«)  and  P(/?).  (Note: 
pro)  and  p (/?)  refer  to  single  test  probability  and  (Of)  and  P(0)  refer  to 
multiple  test  probability.  )  The  determination  of  P(a»  and  P(/3)  considering 
the  above  military  factors  is  not  germane  to  this  effort.  This  effort  is  con¬ 


cerned  with  the  analysis  of  probabilistic  relations  between  error  probabilities 
p(cv)  and  p(/i),  and  standard  deviation  error  of  measurement  instrument , 

standard  deviation  of  test  parameter  parameter  tolerance  for  a  given  ytx 

confidence  IcvqI  (0),  and  decision  limits  (y).  Also  included  are  the  relations 


between  single  test  probabilities  p(o),  p  (/?>  and  multiple  test  probabilities 


P  (o  i ,  and  P  (/3) . 


Wtinn  II.  MATHEMATICAL  MODELS 


1 .  General 

In  the  measurement  of  any  one  individual  parameter  by  the  ETG, 
there  are  three  things' to  be  considered.  The  first  is  the  value  specified  by  the 
weapon  system  for  the  UUT.  This  is  called  the  nominal  value  of  the  parameter 
IN).  The  second  is  the  actual  value  of  the  parameter  <X > .  The  third  is  the 
measured  value  of  the  parameter  (M).  It  is  assumed  that  the  actual  value  (X) 
of  the  parameter  is  related  to  the  measured  value  (M>,  according  to  the  normal 
probability  density  function.  This  assumption  is  based  on  the  fact  that  there  is 
no  inherent  bias  error  as  would  be  caused  by  coupling,  feedback  loops,  and 
switching  in  the  instrument  and  that  all  errors  in  measurement  are  completely 
random  and  normally  distributed.  It  is  also  assumed  that  the  actual  value  X  is 
distributed  normally  about  the  nominal  value  N,  according  to  a  normal 
probability  density  function. 

I 

i 

Since  the  go/ no-go  decision  is  made  on  the  measured  value  of  the 
parameter,  the  normal  probability  density  distribution  for  the  random  measure¬ 
ment  error  is  considered  in  the  following  way.  The  density  function  is  con¬ 
sidered  to  be  centered  at  the  measured  value  M  of  the  parameter.  The  density 

function  with  standard  deviation  cr  describes  the  distribution  of  the  possible 

m 

actual  value  X^'s  that  could  have  resulted  in  a  given  measured  value  M, 


*  2.  Single  Check  Probability  j 

The  normal  probability  density  distribution  for  the  measurement 
error,  for  a  given  value  M-,  has  the  form 

"  i 

"2 

f(X)  =  — - - —  e 
m'/1 7 

The  actual  parameter  value,  X,  is  also  a  random  variable  with  a 
probability  density  function  ffX).  It  is  reasonable  to  assume  that  the  actual  value 
X  is  normally  distributed  about  the  nominal  parameter  value  N  with  a  standard 
deviation  for  the  nominal  parameter  value  of  a  ,  With  no  bias  error  in  the 

measurement  device,  the  measured  value  M  will  also  be  normally  distributed 
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A 


i /-*  V  uiUli  n  rlon  tsifir  ftm/tH  <tn  f  /  ILf  \  If  tha  *n  o  o  an  romonf 


standard  deviation  a  is  an  order  of  magnitude  less  than  the  parameter  standard 
m 

deviation  n  for  the  nominul  value  N  then 
P 


f fM)  «f(X)  . 


Therefore 


-fr)' 


pV2ff 


The  nominal  value,  N,  as  specified  by  the  weapon  system,  also  has 
prescribed  tolerances.  These  tolerances  are  usually  assumed  to  be  ±  nap 

where  it  Is  the  standard  deviation  from  the  acceptable  mean  value  of  the 
P 

parameter.  The  +  nop  value  is  the  upper  specification  limit  and  the  -  n<xp  value 
is  the  lower  specification  limit,  Su  and  respectively.  The  tolerances  may  be 
specified  as  the  nap  level,  where  n  =  1,  2,  3,  5,  etc.  Whatever  the  specified 

no  level,  it  represents  the  allowable  limits  for  the  parameter  values  by  the 

weapon  system  for  proper  operation  of  the  unit  [2], 

*'  i 

In  order  to  assure  that  the  probability  of  an  undetected  defect  does  not 
exceed  a  specified  maximum,  the  measured  value,  M,  must  fall  between  even 
tighter  test  limits; defined  as  upper  and  lower  decision  limits  Du  and  DJP  J 

respectively.  A  go/ no-go  decision  is  then  based  on  whether  the  measured  vaiue 
falls  inside  or  outside  the  decision  limits  and  not  the  weapon  system  specifica¬ 
tion  limits.  The  decision  limits  may  be  set  at  the  specification  limits  or  some 
fraction  of  the  specification  limits.  That  Is: 


( Di)  -  a(V  si) 


where  0  <  a  s  1. 


This  is  shown  graphically  in  Figure  1  |3). 

In  Figure  1,  It  can  be  seen  that  0  =  ±nop  and  y  -  ±a(norp)  rePre®ent® 
the  upper  and  lower  specification  limits  and  the  upper  and  lower  decision  (test) 
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ACTUAL  PARAMCTEi  VALUE 


FIGURE  1.  MEASUREMENT  ERROR  PROBABILITY  DENSITY 
DISTRIBUTION  1 

!  limits.  The  normal  distribution  curve  around  M  is  also  shown  to  clarify  the  , 
proposition  of  setting  the  decision  limits  less  than  the  specification  limits. 

The  probability  of  an  undetected  defect  p (a)  is  the  probability  of 
simultaneously  getting  a  measured  value,  M,  within  the  decision  limits  and  an 
actual  value  X  outside  the  specification  limits.  The  probability  of  getting  a 
measured  value,  M,  within  the  decision  limits  is  f  (M)dM.  The  probability  that 
the  measured  value  resulted  from  an  actual  value,  X,  outside  the  specification 
limits  is 

1  -  / f  (X/M)dX  .  ,  (5) 

The  simultaneous  probability  is  the  product  of  the  two  individual  probabilities: 
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1 


I  11 

1  -  f  f(X/M)dX 

D 


f 


The  probability  of  an  undetected  defect  is  the  summation  of  the  above  probability 
over  all  possible  M's  between  the  declsiop  limits: 


\  u  u 

P<0>  =  f  f (M)  1-  /  f(X/M)dX  dM  . 


The  probability  of  a  false  alarm  p(/J)  is  the  probability  of  simultaneously 
getting  a  measured  value,  M,  outside  of  the  decision  limits  and  an  actual  value, 
X,  inside  the  specification  limits.  Similarly, 


1  U  l  u 

p  Oil  ■  /  f(M)  /  f  (X)dX  dM  +  /  f(M)  /  f(X)dX  dM  .  (•) 


In  equations  (7)  and  (8)  the  limits  are  expressed  aa  follows: 


Dj  =  N-t 


D  =  N+y 
u 


Sj  *  N-  0 


8  =  N  +  9  (•) 

u 

and  f  (X)  and  f(M>  are  as  defined  in  equations  (1)  and  (3)  above. 

Substituting  f(X)  and  f  (M)  into  equation*  (7)  and  (8)  givesan  integral 
equation  which  is,  according  to  Duncan  |4J,  In  the  noncumulative  form  andeaa- 
not  be  integrated  in  closed  form.  Numerical  approximations  have  been  obtained 
and  set  up  in  tabular  form.  However,  in  order  to  obtain  reasonably  close 

^  X  -x 

engineering  estimates  of  p  fo  >  and  p  (/?) ,  an  exponential  of  the  form  e  and  e 
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is  used  and  the  integration  performed  with  the  limits  of  equation  (9)  substituted, 
in  this  manner,  the  p(aj  ana  p(/JJ  equations  are  obtained  in  terms  ot  a  ,  «, 

and  7  (the  desired  parameters)  and  reduce  to  the  following  equations:  , 


p(Q)  = 


a 

m 


2  a  -  or 
p  m 


-1.  15 


e 


(8a  -  ya  +  ya 

P  P  m 

a  a 
p  m 


Equations  (10)  and  (11)  were  solved  parametrically  assuming  that  the 
weapon  systems  specification  limits  fall  at  the  3-slgma  points  ^i.e. ,  0  = 

for  the  actual  parameter  value  distribution  for  a  series  of  values  of  the  following 
ratios: 
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measurement  devtsHrm 

parameter  deviation 


=  accuracy  ratio 


measurement  limit 
specification  limit 


=  decision  ratio. 


The  computed  values  were  plotted  as  function?  of  p(O),  p(/J),  fTm/(Tp> 
and  y/0  .  The  plots  are  shown  to  different  scales  in  figures  2,  3,  and  4. 


3.  Multi  pit  Check  Probability 

The  discussion  up  to  this  point  has  been  concerned  with  individual 
measurement  error  probability.  It  is  often  necessary,  in  the  checkout  of  a 
UUT,  to  make  two  or  more  sequential  tests  on  the  same  unit,  Under  such 
conditions  the  overall  error  probability  becomes  a  function  of  the  number  of 
sequential  tests,  m,  and  the  individual  test  probabilities,  p(rv)'  and  p(/3). 

The  multiple  check  probabilities  P(ct)  and  P  (/?)  may  be  computed  from  the 
following  equations ; 


V  m 

-S  1 

pro  =  i  -  n 

i  -  P(«)<  • 

(12) 

‘  i=  i 

*J 

•  ' 

'  "  m  r 

¥ 

p  (/?)  -  i  -  n 

l-PWI,  . 

(13) 

i«.iL 

, 

where 

m  -  number  of  tests 

pro  2  individual  test  probability  of  undetected  defect 
p(,l)  2  individual  test  probability  of  false  alarm. 

Assuming  that  p(m  and  p  (;))  are  the  same  for  all  m  tests,  then  equations 
(12)  and  (13 1  reduce  to  \ 

P(o>  =  1  -  |  1  -  p(o))m  (14) 

Pi3>  -  l  -  1 1  -  pf  r?>  lm 


(15) 


0.0001  0.0002  0.0003  0.0004  0.0005  0.0004  0,0007  0.0000  0.0007  0.001 
SINGLK  TCST  PROBABIUTY  p  (a) 

.  ,  I 

FIGURE  2.  SINGLE  TEST  PROBABILITY  VERSUS  ACCURACY 
RATIOS  AND  DECISION  LIMITS 


i'  iiti  arm  ii.ji  vuri'  hoivci!  Inr  a  ol  valui'8  ol  |wu  i .  pfn 

anil  rt i  amt  an-  shown  in  plots  of  two  different  scales  in  Figures  .3  and  fl. 

’i  he  nioii  t-nmtnno  wav  of  expressing  measurement  accuracy  is  as  a 

plus  or  minus  percentage  ol  full  scale  reading  with  a  certain  confidence.  The 

standard  deviations  o  or  can  i»e  expressed  in  percentage  when  the  measure- 
re  p 

ments  are  made  at  nearly  lull  scale.  The  relationship  between  a  and  percent  is 
as  follows: 


whe  re 

:Xro  a  accuracy  in  percent  full  scale  j  , 

a  s  standard  deviation 

n  =  desired  confidence  level  (i.e. ,  1,  2,  3,  ...  etc.) 

F.  S.  *  full  scale  deflection  of  instrument. 

'  j 

A  more  complete  and  detailed  discussion  on  the  above  derivation  of  j 

error  probability  density  functions  and  their  relationship  to  test  equipment  may  1 

be  obtained  from  Moon  1 5,  G I .  The  objective  of  this  effort  is  to  apply  the 
mathematical  models  as  shown  to  the  design  criteria  of  the  LCSS-ETG 

equipment.  '  ••  '■  l 
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SINGLE  TEST  PROBABILITT  p  (p)  OR  («) 


SINGLE  TEST  PROBABILITY  p  (p)  OR  p  (a) 


Sect. on  III.  TEST  PLAN 


I  !  '  I'  ll1'  ■  -i  i  111  :  l  si  1 1  n;;  I  lllii  I  .  I  <>  -il  l.i  i  .■  i  u.iilo.i  .".iMipIr  <l! 

;<  i-ii-nls  tn  i-stalilish  estimate*  of  t h» ■  ae-  ir;n  \.  precision,  and  stability 
i>l  liit*  ejfrerem  measuring  and  stimulating  systems  oi  the  l.t  Mis- ETC;  over  the 
fall-scale  ranges  of  the  equipment.  All  values  are.  of  eour.se,  only  sample 
estimates  based  on  limited  samples  of  values  from  only  tv.o  machines.  Care 
should  ho  tal^en  when  Assuming  that  these  t*.vo  machines  art  representative  of 
the  population  of  machines. 


1.  Test  Design 

Two  ETC  sets  were  available  for  use.  In  view  of  the  complexity  of 
the  ETC,  it  was  decided  te  include  In  the  test  design  the  effects  of  time  delay 
between  the  measurement  command  and  the  actual  measurement.  ETC  specifi¬ 
cations  called  for  various  time  delays.  In  order  to  check  for  the  effects  of 
transients  and  drift  upon  short  delays  and  long  delays,  respectively,  it  was 
decided  to  use  the  specified  delay,  and  3  times  and  5  times  the  specified  delay. 
The  effects  of  repetitive  measurements  were  also  considered  by  including  test 
time  durations.  The  durations  were  established  as  four  1-hour  tests  and  one 
4-hour  test  periods  per  machine,  and  delay  times.  The  test  design  1b  shown  in 
Table  11,  from  which  it  can  be  seen  that  each  combination  of  machine,  delay, 
and  duration  is  tested  for  4  hours  for  a  total  test  time  of  48  hours  for  all 
combinations. 

In  order  to  filter  out  as  much  test  environment  bias  as  possible,  the 
test  sequence  was  randomized.  The  randomization  would  tend  to  filter  out 


TABLE  II.  TEST  DESIGN 


Machine 

2 

5 

1  hr  by 
duration 

Tape  delay 

D 

n 

c 

A 

B 

B 

Test  run  hr  freq 

■ 

duration  1  1 

1 

4 

1 

4 

1 

H 

24 

fNom.  hr> 

I 

1  1 

4 

4 

1 

1 

4 

D 

24 

il  hr  by  tape 

8 

s 

... 

H 

* 

s 

■ 

48 

ii  hr  bv 

■ 

M 

H 

HI 

machine 

1 

48 

H'  '-'  i-  '  .1!  i' i<n  i'>  |w  1 1.- !ii  ".or  k  vanatlr  ns,  r.j>eratorn, 

1 ! r; :  1  m|  c ) .1  \  i-fi«i-:s.  •  '  ■  i  hi-  object i\. c  .  (  randomization  is  to  increase  the 

lit  otrilnl  tit  M**jt  t  ;* .  s:  iu  nruiram  »nrl  infriotun  llm  nrrtlvjhillft'  Ihot  InKopont 

'  ,v  11  ■-Mn.r;i!;i!  »na ar*’  eliminated  The  randomization  was  done  using  a 
random  number  generator  coded  for  days,  hours,  machines,  and  delays.  The 
randomized  seqm  r,  <  t-  as  do  ter  mined  hv  the  random  number  generator  is  sluown 
iii  Table  III.  The  machines  are  designated  as  2  and  5  corresponding  to  their 
RCA  serial  numbers.  ETG  ':-002  and  ETG  .’i-005,  respectively.  The  delay 
times  are  designated  as  A.  H,  and  C  for  the  1,  2,  and  5  multiplier  of  specified 
delays,  respectively.  The  numbers  shown  in  parentheses  in  Table  III  refer  to 
the  t.  st  lur.iU-ra  esed  for  bit  ntifuation  in  the  computer  programming.  The 
numbers  run  sequentially  from  1  through  48  corresponding  to  the  total  48  hours 
of  testing.  With  Tables  II  and  III.  all  the  data  can  be  identified,  classified,  and 
grouped  in  any  combination  of  test  conditions  for  analysis. 


2.  T«sf  Method 

The  test  method  w  as  designed  to  simulate  actual  operational  teats  as 
performed  on  a  weapon  UUT  by  the  English  Language  Program  tapes.  It  was 
considered  that  this  method  would  provide  maximum  data  in  a  minimum  amount 
of  test  time.  Also,  it  was  desired  to  leave  the  ETG  sets  available  for  other 
testa  and  uses  as  much  as  possible  during  their  limited  availability  at  Redstone 
Arsenal.  It  was  designed  to  minimize  operator  error,  biases,  and  interpreta¬ 
tion  simulating  actual  UUT  test  conditions. 

A  programmed  semi-automatic  test  tape  was  compiled  to  make  measure¬ 
ments  and  observations  rapidly  and  automatically.  The  tapes  were  identical 
except  for  the  programmed  time  delays  between  stimulus  command  and  measure¬ 
ment  execution.  General  instructions  were  provided  the  operator  for  running 
the  tapes  and  all  measurements  were  performed  and  printed  automatically  by 
the  ETG.  In  programming  the  tapes,  interrupts  were  inserted  with  the  proper 
instructions  to  the  operator  where  manual  switching  Is  required.  It  was 
specified  that  the  operator,  prior  to  running  the  test  tape,  would: 

a  1  visually  inspect  the  ETG  machines  for  defective  or  missing 
components 

b  1  check  out  the  hook-up  and  main  functional  arrangement 

ci  perform  ail  preliminary  operational  checks 

(!■  make  at  least  one  successful  run  on  the  ETG  with  the  RCA  calibra¬ 
tion  ami  maintenance  K’&M)  tape 

c  ■  log  all  unusual  conditions  prior  to  and  during  the  test  tape  run. 


ABLE  III.  RANDOMIZATION  OF  I.OSS  TEST  SCHEDl 
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Wed  27  2 A  (45) 


I  hr  Hl:iml;iins  used  as  ret?  truces  during  the  test  program  wei.  located 
inmd.  o i  adjacent  t.>  the  h  TC.  The  standards  were  prov  drd  and  calibrated  by 
the  r  s  \rrm  Met and  f 'alibi  ation  (Yntei.  Tht  allowing  onuinment  u.:s 
used  ac  stai'da  rds  : 


<i  ) 

D-C  standard'!!  C  different;::)  voltmeter 

UP  7  10R 

b  i 

Kelvin  \  aulev  divide? 

ES!  722 

C) 

Electronic  counter 

HP  i2  liT 

d 1 

Audio  voltage  standard 

Holt  Avs  2215 

C  l 

Ratio  transformer 

Gertsch  PT-2 

f; 

Oscillator 

HP  21 1  A 

S' 

RMS  A-C  differential  voltmeter 

John  Fluke  91!  1  A 

The  semi-automatic  test  tape  has  a  general  test  procedure  to  program 
the  appropriate  ETC  system  to  measure  and  print  the  corresponding  values  in 
the  following  sequence: 

shorted  Input 
external  reference 
self  test  reference 
stimuli 

external  reference. 


This  sequence  provides  data  to  compute  estimates: 


measurement  bias  crroi 


measurement  accuracy  error 


stimulus  setting  error 


all  relative  to  a  known  external  standard  reference.  The  following  ETC 
measurement  parameters  were  written  into  the  test  program  tapes  as  functional 
groups: 

Tost  Operation 

Series  Number  Parameter  Measured 

100  dc  voltage 

2  no  Kesistunee 

.'ton  Frequency 

ton  ae  voltage  (  too  11/ i 

,'ioi'  ae  voltage  t  it)  II/.  i 


Test  Operation 


fiOO  ac  voltage  (10  kHz) 

700  ac  voltage  (1  MHz) 

*00  Pulse  train 


Each  line  on  the  programmed  printout  is  referred  to  as  a  single  observation. 
All  observations  as  printed  out  are  identified  by  their  operation  number  and 
parameter.  On  the  complete  tape  going  through  all  series  (100  through  800) 
there  are  U'l  observations  (i.c.,  measurements).  During  each  hour  of  teat, 
all  measurements  were  replicated  five  times.  Table  IV  is  a  compilation  of  the 
complete  test  tape  sequence  of  measurements  showing  the  parameters,  test 
operations  numbers,  tyoe  test,  ETG  component,  ETG  full-scale  range,  test 
conditions  and  the  expected  measured  value  or  standard  value. 


3.  Test  Meosurements 

A  complete  set  of  the  test  tape  results  are  shown  in  Volume  II, 
Appendix  A.  The  data  shown  therein  have  been  corrected  for  obviously  bad 
data  and  printout  errors.  In  the  great  majority  the  data  are  duplicates  of  the 
original  test  tapes  as  printed  out  by  the  ETG. 
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TABLE  IV.  ETG  TEST  OPERATION  IDENTIFICATION  CHART 


49 

Best  Available  Copy 


o  VII. MS  ■;  7.  (t  V|( MS,  U<o  I!.' 

<•  VHMS  :  ">.  f.:?'  \  I! MS,  inn  11/ 


TABLE  IV.  ETC  TEST  OPERATION  IDENTIFICATION  CHART  .Continued 
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I 
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L 
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Section  IV.  DATA  ANALYSIS 


1.  General 

The  semiautomatic  test  tapes  were  run  on  the  FTC.-.'l  machines  in 
■accordance  with  the  randomized  sequence  shown  ir.  Table  III.  On  the  tapes  are 
H>1  measurement  commands.  Each  tape  was  replicated  five  times  in  each  hour. 
Each  machine  was  tested  for  2 4  hours  fn.  a  total  oi  hours.  Therefore,  the 
total  number  of  observations  is  computed  as  follows- 

X  ~  101  obs'tcp.  ~>  rep/hr  >  41-  hr  .  25.240 

for  all  combinations  of  machines,  time  delays,  durations  and  parameters.  The 
101  observations  are  broken  down  into  a  certain  number  for  each  parameter. 

A  breakdown  of  the  total  number  of  observations  for  any  combination  is  shown 
in  Table  V. 

The  measurements  data  were  printed  out  by  the  ETG-0  printer  on  Taper 
tapes.  The  data  were  coded  for  computer  use,  punched  on  input  cards,  and 
identified  by  test  set  numbers  1  through  48.  The  test  set  numbers  included 
identification  by  machine,  time  delay,  duration,  date  and  hour  of  the  day.  The 
datn  used  are  subject  to  at  least  two  sources  of  error.  The  first  is  machine 
printout  errors.  If  a  measurement  was  different  from  the  last  measurement 
of  the  same  operation  by  less  than  a  factor  of  two,  it  was  included  in  the  analysis 
as  recorded.  If  the  difference  was  greater  than  a  factor  of  two,  the  measure¬ 
ment  was  excluded  from  the  analysis  and  recorded  as  a  machine  "fault"  or 
"dropout. "  All  data  replacing  dropouts  were  estimated  in  accordance  with 
established  missing  value  procedures.  Sample  computations  were  made,  includ¬ 
ing  dropout  values,  to  assess  their  effect  on  a  computed  statistic.  The  factor 
of  two  may  not  be  the  optimum  factor  but  was  selected  to  avoid  any  arbitrariness 
on  determination  of  which  values  to  exclude.  The  second  source  of  error  is  the 
transfer  of  data  from  machine  printout  tapes  to  input  cards.  Obviously  all 
errors  were  not  eliminated  and  the  results  include  the  errors  which  could  not 
be  identified  or  which  were  overlooked.  The  data  used  for  the  computations  are 
shown  in  Appendix  B  of  Volume  II.  Lines  are  drawn  under  values  estimated 
for  dropouts.  The  total  number  of  dropouts  was  54  out  of  24, 240  observations. 

A  11  Bt  of  dropouts  by  operation  number  is  shown  in  Table  VI,  which  also  tabu¬ 
lates  the  dropouts  by  machine,  delay  time,  and  duration  on  an  hourly  basis. 
Shown  for  comparison  are  the  operational  faults  on  the  C&M  tapes  which  occur¬ 
red  at  the  time  the  test  tapes  were  run.  A  description  of  the  faults  is  listed 
below  the  table.  Also  shown  Is  the  ETG-3  parameter  being  tested  bv  each 
operation.  The  800  series  of  test  opwDtions  (pulse  train)  had  the  largest 
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TABLE  V.  SCHEDULE  OF  OBSERVATIONS  BY  MACHINE,  TIME 
DELAYS,  DURATION  AND  PARAMETER 


Machines  1 

ETG3- 

-0002 

ETG3 

-0005 

— 

Time 

2  by 

Delays 

A 

E 

c 

A 

B 

c 

Parameter 

|  Durations  1 

1 

4 

1 

4 

l 

4 

1 

4 

1 

4 

l 

4 

and  Scale 

10  Vdc 

120 

= 

= 

= 

= 

- 

= 

- 

= 

= 

= 

= 

1440 

1 

250  Vdc 

80 

as 

= 

= 

= 

= 

= 

a 

= 

a 

= 

= 

960 

1000  mVdc 

80 

s 

= 

= 

= 

= 

= 

= 

= 

= 

960 

10  k  ohm 
resistance 

80 

3 

3 

= 

= 

= 

= 

25 

53 

— 

55 

960 

100  ohm 
resistance 

40 

480 

1000  k  ohm 

40 

480 

resistance 

« 

o 

1000  kHz 

80 

960 

c 

frequency 

rt 

r. 

u 

10  Vac 

380 

4560 

St 

400  Hz 

V 

£ 

£ 

10  Vac 

260 

3120 

i- 

£ 

50  Hz 

250  Vac 

50  Hz 

340 

4080 

10  Vac 

10  kHz 

120 

J 

1440 

250  Vac 

10  kHz 

220 

2640 

10  Vac 

1000  kHz 

40 

480 

Pulse 

Train 

140 

1680 

IE 

MTD 

2020 

IB 

IB 

IB 

IB 

IB 

IB 

IB 

IB 

IB 

B 

TD 

4040 

= 

= 

= 

IB 

IPfjjl 

24,240 

If 

mmm 

12,  120 

■M 
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AH  IF  VI.  UOM  DILATION  OF  PROGRAM  DROPOUTS  AND  C&M  FAULTS 


number  nf  dropouts  with  32.  Test  hour  48  had  the  largest  number  of  dropouts  by 
tin-  hour  with  13.  A  correlation  of  ^r^prmts  with  C&M  faults  is  evident.  In  the 
table  the  hourly  tent  set  numbers  are  in  the  upper  right  square.  In  test  set  2, 
ope  rut  ion  100  was  a  dropout.  Operation  400  measures  10  VRMS,  400  Hz  ac 
■  nit age.  The  corresponding  C&M  faults,  obtained  from  the  machine  daily  log, 
"•inch  could  he  the  cause,  is  318  and  323  which  checks  the  signal  generator 
frequency.  A  complete  analysis  is  beyond  the  scope  of  this  effort,  but  would 
give  additional  insight  into  test  set  problems. 


2.  Statistical  Computations 

The  main  objective  of  this  program  was  to  determine  an  estimate  of 
the  standard  error  of  measurement  (measurement  standard  deviation)  for  the 
different  parameters  and  scales  of  the  machine.  This  is  referred  to  earlier 
as  n  .  This  statistic  would  be  used  in  conjunction  with  the  weapon  system 

parameter  standard  deviation  to  obtain  an  estimate  of  the  accuracy  ratio. 

This  ratio  is  needed  to  determine  realistically  the  statistical  probability  of 
undetected  defects  and  false  alarms  in  the  go/ no-go  chain.  However,  an  attempt 
lias  been  made  to  estimate  other  characteristics  of  the  ETG-3. 

When  a  measurement  is  made  by  the  ETC-3,  there  are  three  primary 
sources  of  error.  These  are: 

at  inherent  machine  bias  error,  t, 

b 

bi  machine  measurement  accuracy  error,  e 

8- 

ci  machine  stimulus  setting  error, 

To  illustrate,  suppose  the  English  Language  Test  Program  Tape  com¬ 
mands  a  certain  voltage  be  applied  to  the  UUT  with  a  specified  response  to  that 
stimulus  to  be  measured.  The  circuits  providing  that  stimulus  may  be  effected 
bv  a  previous  operation  (bias),  the  measuring  device  may  sense  the  bias  and 
read  erroneously  (accuracy i  or  the  bias  and  accuracy  may  be  acceptable  but 
ilir  voltage  applied  was  not  the  value  specified  (stimulus  setting).  Naturally, 
combinations  of  all  three  may  exist  on  any  given  measurement. 

Estimates  of  bias  error,  r^,  were  obtained  by  programming  a  shorted 

burnt  (SI  i  operation  as  shown  in  Table  IV  and  measuring  for  a  response. 
Estimates  of  measurement  error,  <_  ,  were  obtained  by  measuring  an  accurately 

known  value  from  one  of  the  external  reference  (EH)  standards  mentioned 
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previously  and  subtracting  the  bias  error.  Estimates  of  the  stimulus  setting 
error  were  obtained  bv  proeramrrine  n  stimulus  from  one  of  the  ETG-3 
components  and  subtracting  the  bias  and  measurement  errors. 

The  error  estimates  were  computed  with  the  following  equations: 


N 

E^SI.  -  E 
i  *  1 

%  =  - N -  * 

where 

SI  £  shorted  input  value 
EV  s  expected  value 
N  £  number  of  responses. 

S  (eR.  -  Ev) 

_  1  =  1 _ 

Ca  “  N  "S’ 

where  ER^  a  external  reference  measured  value. 


(17) 


(18) 


-  EV  ) 

i  -  1 


(19) 


where  MV  a  measured  value  of  ETG-3  component  (e.g, ,  DA-1,  DC-1,  STR). 
The  above  computed  errors  are  the  means  of  the  differences  between  the  actual 
(or  expected)  value  and  the  measured  (or  unexpected)  value. 

The  standard  deviation  of  measurement (  a  \  is  computed  by  the  follow¬ 
ing  equation:  '  ' 


N  _ 

1 1  (MV  -  EV)  -  (MV  -  EV))! 

s  i_LJ _ 

M  N  -  1 


(20) 
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which  is  the  measurement  variance.  The  standard  deviation  is  simply  the 


I  VMHI 


jut  mt  equation  anu  i»  uetsigimieu  o  rtiuier  man  u  since  n  is 
•  •om|i'itefl  from  a  limited  sample  of  data. 


The  statistics  as  computed  with  the  above  equations  were  computed  in 

t'\n  vnva.  The  first  computations  were  made  for  each  hour  ftest  set)  of 

testing.  The  purpose  of  this  was  to  use  the  estimates  of  these  statistics  as 

response  variables  for  a  subsequent  analysis  of  variance.  The  analysis  of 

variance  enables  one  to  test  for  significant  differences  between  factors.  In 

addition  i'<  '.sting  for  significant  differences  between  the  main  iactors  (i.e., 

machines,  time  delays  and  durations),  -tests  can  also  be  made  for  differences 

between  second  order  or  interaction  effects  (e.g.  ,  machines  x  time  delay 

effects)  as  well  as  third  order  interaction  effects.  The  second  set  of 

calculations  v  as  marie  using  all  observations  for  each  parameter.  These  are 

referred  to  as  the  overall  values  of  >,  .  e  ,  r  ,  and  S...  These  calculations 

b  a  s  M 

are  more  sensitive  to  subsequent  tests  due  to  the  much  larger  sample  size 
and  greater  degree  of  freedom. 


The  hourly  computations  are  shown  in  Volume  II,  Appendix  C.  These 

values  were  computed  with  small  sample  size  and  degrees  of  freedom  and  are 

used  only  in  the  analysis  of  variance  computations.  The  overall  statistics  are 

shown  in  Table  VII.  While  the  statistics  c.  ,  e  ,  and  «  give  some  indication 

V  a  8 

of  the  machine  error  by  parameter  and  scale,  the  measurement  standard 
deviation  is  the  important  statistic  from  the  standpoint  of  machine  variability 

and  determination  of  statistical  control  of  desired  probability  levels  of 
"undetected  defects"  and  "false  alarms,"  p (or )  and  p (/?), 


3.  Analysis  of  Vorianct  (ANOVA) 

The  hourly  computations  of  c.  ,  e  ,  e  ,  and  8,, 2  were  used  as 

b  a  s  M 

response  variables  for  analyses  of  variance  of  all  parameters  and  scales. 
Hence,  there  were  4  x  14  =  5G  analyses  performed.  The  statistical  models  are 

•.  (i.  ’  k.  1)  -  p  +  A  +  B.  +  C:  +  AB..  +  AC..  +  BC„  +  ABC...  +  e,  . 
i>  i  j  k  ij  ik  jk  ijk  l(ijk) 

(21) 

ami  similarly  for  <  ,  «  ,  and  S.A  This  model  assumes  that  each  response  of 
a  s  M 

errors  ■ .  ,  -  ,  r  and  variance  S.,’  is  the  algebraic  sum  of: 
b  a  s  M 
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TABLE  VH.  COMPILATION  OF  OVERALL  COMPUTED  ERRORS,  STANDARD  DEVIATION  AND 
PERCENTS  BY  PARAMETER  AND  SCALE 


a  universal  mean  of  the  error  m  (i.e. ,  the  true  error  or  variance 


In  a  machine  effect  on  the  error  or  the  variance,  C', 

K 

c  a  time  tie  lav  effect  on  the  error  or  the  variance, 

A  a  f!  ir  ftir'u  effect  on  the  error  or  the  variance,  A 

i 

c'  an  interaction  effect  on  the  error  or  the  variance,  \B  .,  AC 

BC  .  ABC.  ,  lj  Ik’ 

in  nk 


r-nvoin  .'erficmal  i -:|i',i’ini(>ntal  '•yrer,  e 


1  (ijki ' 


Sinci'  the  model  is  t'l.seil,  none  of  the  effects  can  be  determined  absolutely. 
Tlirv  can  Be  measured  c  nlv  as  differential  deviations,  i.e.  : 


a  ■  the  A.  etfects  as  deviations  from  n 
1 

In  the  IV  effects  as  deviations  from  // 
ci  the  C,  effects  as  deviations  from 

l\ 

d»  the  A IV,  AC,,  ,  BC,,  as  deviations  from  the  A.  +  IV,  A,  +  C,  , 
ij  ik  jk  l  J  i  k 

anti  B^  +  respectively 

c>i  the  ABC,.,  as  deviations  from  A,  +  B  +  C,  . 

ijk  i  j  k 

Because  of  the  largo  volume  of  data  and  the  number  (5(5)  of  analyses  required 
to  cover  all  combinations  of  parameters  and  statistics,  the  analyses  were  com¬ 
puter  programmed.  The  program  used  was  obtained  from  Edwin  Bartee  and 
was  compiled  by  J,  A.  Svestka.  A  printout  of  the  computer  program  is  shown 
in  Volume  II,  Appendix  D,  including  dimensions  and  correspondence  state¬ 
ments  and  subroutines.  Each  combination  of  machines,  time  delays,  and  dura¬ 
tion.  of  which  there  are  12  (2  x  3  x  2),  had  four  responses  in  each  cell.  Since 
each  hour  of  test  was  replicated  five  times,  four  responses,  which  represents 
i  hours  of  testing,  was  the  average  value  for  that  hour.  Table  VIII  is  a  com¬ 
pletely  coded  layout  of  the  data  input  to  the  analysis  of  variance.  The  table 
Identifies  the  factors  in  the  mathematical  model  [Equation  (21)  ] ,  the  test  set 
numbers,  the  date,  and  the  hour.  To  illustrate,  the  four  response  values  for 
the  cell  representing  a  1-hour  test  duration^  A^,  with  the  specified  delay  time 

(iv),  on  the  ETG-3-0002  machine  are  data  test  sets  2,  12,  45,  and  47 

which  arc  coded  as  1,  2,  3,  and  4  for  input  to  the  computerized  analysis  of 
variance.  A  computer  printout  of  the  input  data  to  each  anulysls  of  variance 
nnd  the  \\'(A’A  results  arc  shown  in  Volume  II,  Appendix  E. 


Test  IXiration  (A.) 


TABLE  VIII.  COMPILATION  OF 
DURATION,  DAY  AND  HOUR 


Machines 


'<V 


ETG:i-(mn2 


Tap. 

S  lit  I 

j 

1 

:\ 

> 

2  1 

3  9 

1  17 

2/  H-11 

2/  1:1-11 

2/  13-10 

12  2 

,  Id 
30 

18 

11 

1  hr 

2/2:1-11 

2/21-11 

2/23-9 

la  '* 

1,  11 

2.7  19 

<" 

.‘1/27-14 

3/20-13 

3/11-10 

c 

0 

PS 

47  4 

48  12 

40  20 

u 

A 

2/29-9 

3/29-12 

3/28-12 

"8 

& 

h 

21  ° 

4  13 

13  21 

2/08-10 

2/  16-8 

3/01-8 

22  6 

-  14 

22 

10 

■1  hrs 

2/08-11 

2/10-9 

3/01-9 

23  7 

0  15 

23 

17 

3/08-12 

2/10-10 

3/01-10 

24  8 

7  16 

18  24 

3/08-13 

2/10-n 

3/01-11 

I  Tapes 
1'  Machine 


8 


21 


8 


8 


TESTS  BY  MACHINE,  TAPE, 


V 

ETG.'j-OOf). 

Dm 

1 

- 

,) 

tioi 

23 

S 

33 

19 

11 

2/  19-  13 

3/01-  14 

2/20— ?t 

13  *'■ 

20 

2/29-11 

3/04-  12 

2 

27  27 

33  'U 

20 

3/  13- 12 

3/18-9 

3/  128 13 

28 

32 

43  36 

38  " 

3/  13-9 

3/21-10 

3/  19-  13 

29 

28 

39  37 

34  45 

1 

3/  14- 10 

3/20-8 

3/18-10 

Mg 

40  38 

35  49 

3/20-9 

3/18-11 

2  1 

„„  31 
30 

41  39 

30  47 

3/  11-  12 

K> 

O 

1 

» — » 

O 

3/  1»-12 

32 

31 

40 

42 

37  48 

3/14-13 

3/20-11 

3/  18-13 

1 

8 

8 

8 

RH 

24 

I 

sr  for  analysis  of  variance  flats  input 
intification  number 
in  1908 

t  (c.  g.  1400  or  2  pm  I 


4.  Test  of  Hypotheses 

A  statistical  hypothesis  is  an  assumption  about  the  population  being 
sample*:.  It  usually  consists  of  assigning  a  value  to  one  or  more  parameters 
of  the  population.  A  test  of  a  hypothesis  is  simply  a  rule  by  which  a  hypothesis 
is  either  accepted  or  rejected.  The  rule  is  usually  based  on  sample  or  test 
staiMies  used  to  test  the  hyootheses.  The  critical  region  of  a  test  statistic 
consists  of  all  values  ol  the  test  statistic  a  here  the  decision  is  made  to  reject 
or  accept  the  hypothesis.  Sir.ec  hypothesis  testing  is  based  on  observed  sample 
statistic?  computed  on  N  observations,  the  decision  is  always  subject  to  errors. 
Ti  tit.  h\  polar  -is  is  r-nily  l rue  :.n  .  is  rejected  by  the  sample  statistic,  a  Type  1 
error  is  committed.  The  probability  of  a  Type  1  error  is  designated  as  o.  If 
the  hypothesis  is  accepted  when  it  is  not  true,  i.e.  ,  if  some  alternate  hypothesis 
is  true,  then  a  T\po  II  error  has  been  made.  The  probability  of  a  Type  II 
error  is  designate*!  J. 

Tin*  overall  values  of  *,  ,  »  ,  and  -  for  all  parameters  were  tested 

b  ;t  s 

with  the  I  allowing  hypotheses  l\.r  the  sample  error  statistics: 


Ho,  =  (! 

-V  --  0. 

05 

Ho-,  r  ~  0 

o  =  0. 

or, 

a 

Ho3  i  *-•  o 

o-o. 

05 

The  alternate  hypotheses  arc: 


HU:  •  *  0 

a 

III,:  ■  *  n  . 

s 

The  computed  estimates  of  the  errors  are  the  differences  in  means. 

The  hypothesis  (Hoi  is  that  the  errors  arc  not  significantly  different  from  zero. 
This  is  based  on  the  assumption  that  the  universe  mean  of  the  errors  is  zero. 

Ii  the  basic  hypothesis  is  rejected  |at  the  fl  -  O)  level  of  confidence)  the 
alternate  <H,:i  is  accepted.  However,  if  the  basic  hypothesis  is  accepted  at  a 
specific*!  level  of  confidence  there  is  still  a  chance  that  an  error  of  the  first 
Kind  has  been  made.  In  testing  hypotheses  pertaining  to  the  universe  mean 
the  ptoecdure  is  simplest  if  the  standard  deviation  of  the  universe  is  known. 

In  this  case  the  sample  errors  are  treated  as  having  a  normal  distribution  with 


bl 


Best  Available  Cop 


a  mean  equal  to  the  universe  mean;  but,  the  universe  standard  deviation  is 
uh'kium' ii.  Hence,  Lhe  correct  test  statistic  is  the  T  statistic.  The  M’ 
statistic  is  used  when  the  standard  deviation  must  be  estimated  from  the  sample 
data.  The  *t*  statistic  is  computed  an  follows: 


’t’ 

to  I 


X  -  X’ 
S/vN 


where 

X  calculated  mean 
X’  '  universe  mean 
N  2  sample  size 
o  =  risk  =  0.  03 

and 


S  = 


123) 


If  the  computed  value  of  *t*  is  less  than  the  table  value,  for  N  -  1  degrees  of 
freedom  and  a  -  0.05,  then  the  basic  hypothesis  is  accepted  and  the  alternate 
is  rejected.  A  computer  printout  of  the  results  Is  shown  In  Volume  II, 
Appendix  F.  A  summary  is  shown  in  Table  IX  in  the  row  labeled  "T  Test  of 
Overall  Values.  " 


In  the  ANOVA  the  basic  hypotheses  are  as  follows: 

Ho:  A  =  0  a  =  0.  10 

Ho:  B.  =  0  tv  =  0.  10 

*  j 

Ho:  C,  =  0  a  *  0. 10 
k 

and  similarly  for  the  second  order  (ABij,  etc. )  effects  and  the  third  order 
fABCijk)  effects.  The  alternate  hypotheses  are: 
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TAPLE  !X.  COMPILATION  OF  RESULTS  OF  TESTS  OF  HYPOTHESES  ft*  TEST  AND  'F'  TESTS' 


I 


Y  '’i~w 

H  •  R  *0 
I  1 

Hl:  V° 

and  similarly  for  the  second  and  third  order  effects. 

The  objectives  of  basic  hy  pothesis  tests  are  to  determine  that  the  main 
factors  of  duration  time  delay  ^  R  .  j  and  muchines^C^  ^as  well  as  inter¬ 

actions  between  main  factors  ^c.g.  ,  ADj.^do  not  significantly  effect  the  response 

variables,  e.  ,  c  .  c  ,  and  S"  and  they  are  essentially  zero,  i.e. ,  there  is  no 
Das 

treatment  effect.  The  test  statistic  is  the  F  distribution  which  is  the  ratio  of 
two  independent  chi-square  distributions.  This  means  that  the  F  distribution 
is  the  ratio  of  the  mean  squares  between  treatments  to  the  mean  squares  (Mb) 
within  treatments  or  mean  square  for  error. 


where 


_  MS  (treatments) 
F(fi)  MS  (error) 


MS  (treatments) 


sum  of  squares 
degrees  of  freedom 


MS  (error)  = 


_ sum  of  squares 

degrees  of  freedom 


(24) 


(25) 

(26) 


If  the  computed  value  of  F  is  equal  to  or  greater  than  the  table  value  of  F  for  the 
set  level  of  confidence  ((v )  and  the  proper  degrees  of  freedom  then  Ho:  is 
rejected  and  HI:  is  accepted.  A  summary  of  the  results  of  the  ANOVA  Is  shown 
in  Table  IX.  The  ANOVA  printouts  for  each  statistic  and  parameter  are  shown 
ir;  Appendix  G  of  Volume  II. 


) 
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Section  V.  DISCUSSION 


i  bis  analysis  has  been  primarily  concerned  with  the  estimation  of  the 
measurement  variance  and  secondarily  with  the  bias  error,  measurement  error 
and  stimulus  setting  error.  The  estimation  of  these  statistics  has  been  done 
with  a  sample  of  data  taken  from  two  machines.  The  analyses  have  also  esti¬ 
mated  the  effects  of  time  delays  and  duration  of  tests  on  these  statistics.  The 
statistics  were  computed  for  fourteen  different  measurement  parameters  and 
scales  performed  bv  the  ETG-3  test  sets.  These  parameters  are  not  all  of  the 
functions  of  the  machines  and  represent,  at  best,  only  a  partial  test  program. 
The  results  represent  estimates  for  a  small  sample  of  machines,  factors,  and 
measurements,  and,  should  be  accepted  as  such.  The  results  shown  in 
Tables  VII  and  IX  will  be  discussed  briefly.  The  discussion  will  be  categorized 
by  machine  parameters  which  are  probably  of  greater  interest,  to  machine  users 
than  the  sample  statistics. 


1 .  10-Volt  dc  Scole 

In  Table  Vll,  the  estimated  value  of  the  bias  error  is  0.000940,  the 
accuracy  error  is  -0.  00197  and  the  stimulus  Betting  error  Is  0.  0010f>,  The 
standard  deviation  Is  0.  02515.  Table  IX  shows  that  the  hypothesis  that  the  bias 
error  is  not  significant  Ho:^  =  0  is  rejected.  The  hypotheses  that  -»  0 

and  “  0  were  accepted  In  the 't'  tests.  In  the  ANOVA  portion  of  Table  IX,  it 

is  shown  that  machine  effects  were  significant;  i.e. ,  Ho:  -  0  was  rejected 

for  both  statistics  and  rg.  In  Table  VII,  the  3S  level  of  standard  deviation 

expressed  as  percent  full  scale  is  0.  8  as  compared  with  the  specified  (assumed 
3S)  value  of  0.04  percent,  This  ts  a  factor  of  20  higher  than  specified  assuming 
the  specified  values  are  correct. 


2.  250-Volt  dc  Seal* 

For  this  parameter,  the  hypotheses  that  and  c  =  0  were  rejected 

Indicating  that  there  are  significant  bias  r.nd  stimulus  errors  on  this  parameter. 
The  ANOVA  reveals  that  the  contributing  factors  to  and  e  are  main  effects 

of  delay  time  and  machines  and  interactions  between  delay  times  and  machines. 
The  estimated  values  as  shown  1;  Table  VII  are  e^  =  0.  147,  f&=  -0.0798, 

(  =  1.  77  and  S  =  2.  975.  The  3S  level  expressed  as  a  percent  =  3.6  as  compared 
s 

with  the  specified  value  of  0.08  percent,  a  factor  of  about  45  greater. 
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3.  1000-Millivolt  dc  Scale 

The  sample  of  data  reveals  that  all  errors  were  significantly  differ¬ 
ent  from  zero.  The  ANOVA  indicates  that  all  factors  significantly  contribute  to 
bias  error.  Delay  times,  machines  and  delay  time-machine  interactions  contri¬ 
bute  to  r  .  All  factors  except  durations  contribute  to  (  and  delay  times  and 
a  s 

machines  contribute  to  the  variance,  S2.  The  estimated  values  in  Table  VII  are 

f,  =  0.349,  c  =  -3-46,  €  »  0.476,  and  S=  2.40.  The  3S  level  is  0.7  percent 
b  a  s 

as  compared  to  the  specified  value  of  1. 2  percent. 


The  rest  of  the  parameters  are  left  to  the  reader.  An  overall  look  at 
Table  VII  shows  that  the  machines  have  met  the  specified  standard  error  of 
measurement  on  the  1000-millivolt  dc  scale,  the  10-kilohm  resistance  scale, 
the  1000-kilohm  resistance  scale  and  the  10-volt  PP  ac  voltage  scale  at 
5  1-megahertz  frequency.  The  100-ohm  resistance  scale  is  close  with  2  percent 
as  compared  with  1.  73  percent  specified.  The  10-volt  ac  400-hertz  scale  is 
not  good  with  a  19  percent  as  compared  with  0.  3  percent' specified.  The  worst 
seems  to  be  the  1.0  megahertz  frequency  parameter  with  0.5  percent  as  com¬ 
pared  with  0. 0011  percent  specified  a  factor  of  about  500  greater. 


Table  IX  shows  the  bias  error  was  significant  for  all  parameters  and 
scales.  Accuracy  error  was  significant  for  all  parameters  except  10  volts  dc, 
250  volts  dc,  and  1000-kilohertz  frequency,  i.e. ,  11  out  of  14.  Stimulus  setting 
error  was  significant  on  all  parameters  except  10  volts  dc,  1000-kilohertz 
frequency  and  10  volts  ac,  50  hertz,  i.  e. ,  11  out  of  14. 


The  ANOVA  portion  of  Table  IX  shows  that  test  duration^  Aj  jwas  detected 
as  a  significant  effect  on  the  responses  of  c.  ,  (  ,  c  ,  and  S!  in  only  4  analyses 

O  a  8 

out  of  56.  The  main  effect,  delay  time  ^B^was  found  to  be  significant  21  times 


out  of  56.  The  main  effect  of  machines 


(c0 


was  found  to  be  significant  46 


times  out  of  56.  The  second  order  effect,  AB^,  6  times;  AC^,  13  times; 

BC  ,  11  times;  and  ABC  ,  8  times  out  of  56.  Machine  effects  far  out- 
jR  ljK 

weigh  the  other  main  and  interaction  effects. 


The  estimates  of  the  standard  deviation  of  measurement  as  obtained 

from  this  sample  will  be  used  as  an  estimate  of  a  .  Previously,  the  use  of  <r 

m  m 

in  the  determination  of  an  accuracy  ratio  was  discussed.  Also,  the  use  of  the 
accuracy  ratio,  decision  limits  and  p(n )  and  pf/b  in  the  determination  of  the 
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statistical  capability  of  the  test  sets  for  single  tests.  With  the  values  of  o 

m 

as  obtained  a  few  examples  will  be  made  with  actual  values  of  the  weapon  system 
components  tolerances^  (T^^.  Also,  decision  limits  will  be  assumed.  The 


parameters  1000-millivolt  dr  scale,  10-kilohm  resistance  scale,  1000-kilohm 
resistance  scale  met  the  MIS-6000  specifications.  These  parameters  and  their 
values  are  used  in  comparison  with  misBile  specification  values  for  components 
which  would  require  testing  on  these  scales.  Also,  one  ol  the  parameters  which 
did  not  meet  MIS-6000  specs,  the  10-volt  dc  scale,  is  included  lor  comparison 
purposes.  'I he  sample  estimated  values  of  the  standard  dcwiatlor 


of  these  parameters  is  divided  by  the  component  standard  deviation (  a  \ 

/  \  V  p/ 

obtain  an  accuracy  ratio!  a  i  a  ].  Decision  limit  ratios  (v/flj  of  1.00, 

\  ?t>  p/ 


if  )  for  each 
»y 


to 

0.1)5, 


and  0. 1)0  were  assumed  tor  this  example. 


The  values  for  each  weapon  system  are  actual  vnluos  obtained  from  the 
system  specifications.  The  TOW  system  tolerances  arc  based  on  2a  levels  and 
the  SHILLELAGH  .system  tolerances  are  based  on  Her  levels  The  values  of  n 
for  both  systems  were  obtained  from  the  system  tolerances  accordingly.  The 
values  of  pfoj  and  p(/J)  were  obtained  from  the  computer  printouts  used  to 
plot  Figures  2,  3,  and  4.  The  results  are  shown  In  Table  X. 


The  system  components  are  as  shown  In  Table  XI.  Similar  analyses 
can  bo  made  for  other  parameters  and  scales  and  weapon  systems. 

Table  X  demonstrates  the  relationships  between  accuracy  ratios, 

decision  limits,  undetected  defects  and  false  alarms  for  single  tests  for  actual 

system  components.  It  also  demonstrates  the  trade-offs  and  compromises 

available  to  hold  certain  p{n)  and  p  (/?) .  A  component  of  Shillelagh  has  a  50- 

percent  tolerance  of  measurement  on  the  1000  millivolt  dc  scale  which  gives  an 

accuracy  rutlo  of  0.  014  oi  about  70  to  1  when  compared  with  sample  o  found 

m 

for  the  ETC  on  that  scale.  Therefore,  the  probability  is  that  practically  no 
undetected  defects  will  get  by  the  tests;  however,  some  false  alarms  varying 
from  1  out  of  1000  to  1  out  of  100  depending  on  the  decision  limits  will  probably 
occur.  An  accuracy  ratio  of  0.  504  (about  2  to  1)  shown  for  TOW’  component  on 
10-volt  dc  scale  will  result  In  about  1.5  units  out  of  100  at  decision  limit  of 
1. 00  to  1.  1  units  out  of  100  at  decision  limit  of  0.  90  passing  with  "undetected 
defects.  "  Similarly,  5.  o  and  6. 6  "false  alarms"  will  occur  out  of  100  testa. 


6  8 


,  TABLE  X.  SINGLE  TEST  p(<v)  AND  p(H)  FOR  RATIOS  OF  CALCULATED  ETC.  t  AND  SYSTI  7\  .  ,'ECIFIED  rr 


■v 

I 

I 
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Section  VI.  SUMMARY 


This  analysis  effort  has  resulted  in  estimates  of  bias  error,  accuracy 
error,  stimuli  setting  error  and  overall  standard  deviation  of  measurement 
(ETG  precision)  for  13  different  combinations  of  parameters  and  scales  that  the 
test  set  is  capable  of  measuring.  The  sample  statistics  are,  of  course,  subject 
to  sampling  errors,  hidden  or  undetected  effects  and  to  human  errors  in  the 
test  program  and  the  computations.  The  test  program  could  be  improved  and 
the  statistical  computations  expanded  for  a  more  complete  and  detailed  analysis 
which  may  reduce  some  of  the  inherent  errors  in  the  test  program  and  computa¬ 
tions.  However,  the  results  found  from  this  sample  of  data  give  good  indica¬ 
tions  of  the  following  conditions  of  the  ETG-3  test  sets. 


a»  There  is  significant  bias  error!  e  Ion  all  parameters  and  scales 
tested.  '  °  ' 

b)  There  is  significant  accuracy  error  ^  jon  11  out  of  13  parameters 
and  scales.  (10  Vdc,  250  Vdc,  and  1  MHz  frequency  accepted). 

c)  There  is  significant  stimulus  setting  error  ^  on  10  out  of  13 

parameters  and  scales.  (10  Vdc,  1  MHz  frequency,  10  VRMS, 

50  Hz  ac  excepted) . 

d)  A  significant  effect  of  the  main  factor  test  duration  jwas 
detected  on  two  out  of  13  parameters  and  on  s2  two  out  of  13 
parameters. 

e)  A  significant  effect  of  the  main  factor  delay  time  ^  was  detected 
on  nine  out  of  13  parameters,  on  three  out  of  13,  on  seven 
out  of  13,  and  on  s2  two  out  of  13, 

f)  A  significant  effect  of  the  main  factor  machines  was  detected 

on  e,  12  out  of  13,  on  t  ten  out  of  13,  on  c  13  out  of  13,  and  on  s2 
b  a  s 

11  out  of  13. 


g)  The  ETG  is  meeting  the  MIS  6000  specification  for  precision  on  only 
four  parameters  out  of  13  considered  or  about  30  percent. 

ht  On  the  basis  of  the  weapon  system  tolerances  used  In  the  sample 
comparisons,  the  ETG  will  not  be  able  to  hold  the  specified  "across 
the  board”  1  out  of  100  probabilities  for  single  checkouts  except 
where  the  system  tolerances  are  broad  resulting  in  a  high  accuracy 
ratio.  Multiple  sequential  test  probabilities  will  be  worse. 
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i)  The  present  situation  of  incompatibility  is  between 

li  across-the-board  1  out  of  100  probabilities 

2  :>  MIS  0000  specified  LCSS  precision 

2 1  unreasonably  close  tolerances  for  weapon  system  UUT's 

il  English  Language  Test  Program  requirements  which  must  be 
resolved  in  a  comprehensive  manner. 

The  pulse  train  is  not  included  In  the  parameters  an  r.o  suitable  specified 
capability  for  the  pulse  train  was  found  and  also  due  to  the  excessive  percentage 
of  the  total  dropouts  (n0  percent)  found  on  the  pulse  train  portion  of  the  test 
program. 

The  test  set  functional  dependence  cannot  be  commented  on  even  though 
it  was  considered  to  have  had  only  f>6  command  dropouts  out  of  24,  240  commands; 
future  tests  could  possibly  obtain  an  estimate  of  functional  dependence  in  light 
of  the  presently  obtained  estimates  of  precision.  That  is.  the  factor  of  2 
variation  from  an  expected  value  used  in  this  analysis  could  now  be  tightened 
up  to  obtain  a  better  estimate  of  functional  dependence.  This  would  probably 
lower  the  variability  and  functional  dependence. 

A  more  complete  test  program  Is  considered  highly  desirable.  It  should 
be  performed  on  a  continuing  basis  by  the  LCSS  Project  Office  or  the  prime 
contractor.  Tests  should  be  conducted  In  the  field  to  obtain  estimates  of  the 
effects  of  other  factors  such  as  temperature,  humidity,  pressure,  dust,  etc. , 
on  the  ETC- 3  performance.  Such  a  program  to  be  properly  executed  would 
require  a  considerable  level  of  effort;  but,  it  would  be  a  notable  achievement 
and  contribution  iri  the  area  of  evaluation  of  the  test  capability  of  complex  test 
equipment, 
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ESTIMATION  OF  LAUNCH 


VEHICLE  PARAMETERS  FOR  THE  GIVEN  MODEL 

0,t 

y  =  9,0  sin  (03t) 

John  W.  Howerton  and  D.  Ray  Campbell 
Redstone  Arsenal,  Alabama 

SUMMARY 

This  paper  presents  analysis  techniques  of  a  launch  vehicle  of  an  antitank 
missile  system.  Since  the  development  of  a  mathematical  model  from  an 
analysis  of  components  subsystems'  responses  must  be  checked  against  overall 
performance,  it  was  assumed  that  the  vehicle  responded  as  a  second  order 
differential  equation.  The  solution  of  this  equation 'is  fitted  to  the  experimental 
data. 

The  parameters  0,,  0j,  03  are  estimated  for  the  model  y  «  0ieOj  sin  (03t) 

V  Yh^’  **  “  1»  2  . . .  N.  Several  techniques  of  estimation 
are  used.  The  following  methods  are  included: 

(1)  Prony's  Exponential  Approximation 

(2t  Least  Squares  Polynomial  —  Taylor  Series 

(3)  Differential  Correction 

(3)  Gradient- Descent 

(5)  Modified  Newton-Gauss. 

A  comparison  of  the  techniques  is  presented  and  a  "best"  method  of 
estimation  is  selected. 

This  article  has  been  reproduced  photographically  from  the  authors'  manuscript 


and  given  data  points  [' 
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Preceding  page  blank 


BACKGROUND 


During  the  investigation  of  a  particular  antitank  missile  system,  it 
appeared  that  launcher  motion  may  have  an  effect  on  the  missile  trajectory. 

Thi«  phenomenon  results  from  the  fact  that  the  missile  is  commend  guid'd  no 
•ho  tracker  ts  mount  ml  rigidiv  on  the  launcher.  The  program  require.?  :  r  ;  •, 

•n  th'-!  h»  o'.  \  •!•  ;»>d  f.  r  evimnf»nj»  th<-  inr-rtial  restoring,  and  damping  p-‘  ■  r  • , 
•«f  a  vchin •  !»•  from  measured  motion  of  the  vehicle  during  the  launch  phase  Sm 
flight.  If  will  then  he  po\=s;)le  to  determine  more  readily  if  the  vehicle,  r:  <  •:  »:■ 
desired  performance  requirements  and  to  trace  any  degradation  which  r.u  -  ••  ■  u 
it! or  r:  number  of  hours  of  use  in  the  field. 

The  mathematical  model  utilized  for  this  investigation  is  a  damped  :>ine 


nave  whic  h  comes  ir  -m  the  solution  of  a  second  order  differential  equation.  For 


the  purpose  of  this  investigation  the  mathematical  model  has  been  assumed  to  be 
correct  and  is  of  the  form:  y  =  P:e°^  sin  (03t) . 


Best  Available  Copy 


I.  INTRODUCTION 


If  a  scatter  diagram  in  the  x,y  plane  indicates  that  a  straight  line  will  not 
fit  a  set  of  points  satisfactorily  because  of  the  nonlinearity  of  the  relationship, 
it  may  be  feasible  to  fit  a  simple  curve  that  will  yield  a  satisfactory  fit.  Since 
an  investigator  always  strives  to  explain  relationships  as  simply  as  possible, 
with  the  restriction  that  his  explanation  be  consistent  with  previous  knowledge, 
he  will  prefer  to  use  a  simple  type  of  curve.  It  follows,  therefore,  that  the  type 
of  curve  to  use  will  depend  largely  on  the  amount  of  theoretical  information  one 
has  concerning  the  relationship  and,  also,  convenience. 

In  the  problem  under  study,  it  was  assumed  that  a  second  order  differen¬ 
tial  equation  described  the  motion  of  the  vehicle.  The  data  recorded 

Ih  *  1,2,...  n|  ,  which  was  the  displacement,  y^,  from  equilibrium 
position  at  time,  t^.  Thus,  the  solution  to  the  differential  equation, 

Y  *  sin  03t  (1) 

is  to  be  fitted  to  D  by  determining 


so  as  to  minimize  the  residuals. 

An  attempt  Is  made  to  solve  directly  the  least  squares  problem  resulting 


from  (1)  and  D.  Let 


Q(0) 


-  ete  sin  03th 


(2) 
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By  taking  partial.*  * >f  QlOi  with  respect  to  each  of  the  parameters  and  setting 


the  rv.-inUs  epuu:  I  ■  /.o i1",  the  following  nonlinear  system  results: 


'iQfii  > 
rlf1 1 


N  2/)-t, 

V  “  h  . 


''i  c' 
h  =  I 


N  0,t. 
V'  ..  "h 


rKj(  it, 

7m, 


N  :’a., t 


N  o2t. 


-r,  t,;'  h  S'H2  (oath)  +  i  yhthe  h -^r,  (M,.'  « 

n  -  l  ii=  l  ■  ' 


N  20,i 


N  ^ 


2|!fi  -  V  the  ",lr  CO,  (o.th')-  l  VV  h  «"  (^h)  ’  • 

h - 1  '  ■  h=l 


(3) 

t 

Although  the  above  system  can  be  reduced  to  a  two-dimensional  system,  the  two- 
dimensional  system  indicated  that  the  direct  approach  ia  not  feasible. 

To  tnke  a  slightly  different  approach,  the  Laplace  transform  of  Q(0)  was 
taken.  The  partial  derivatives  of  the  resulting  expression,  I,fQ[(  0)  ,  t]  j ,  with 
respect  to  each  of  the  three  parameters  we^e  taken  and  the  results  set  equal  to 
zero.  The  exponential  ami  the  sine  function  are  suppressed  in  the  resulting 
system,  but  the  ratioaal  functions  involved  proved  unmanageable  and  this 


approach  was  also  abandoned. 

Also  attempts  at  simplification  by  taking  the  logarithm  of  (l)  and  (11) 


proved  1'itile. 

Since  the  direct  approach  to  the  problem  would  not  yield  a  solution, 
sever  il  ndirr.'ct  methods  we  re  applied.  These  methods  were: 


/x 


* 

(1)  Proney's  Exponential  Approximation 

(2)  Taylor  Series  —  Least  Squares  Polynomial 

(3)  Modified  Newton-Gauss 

(4)  Steepest  Descent  —  Method  of  Optimum  Gradients 

(5)  Differential  Correction. 

Based  on  machine  time,  number  of  iterations  and  minimum  of  residuals, 
some  methods  gave  better  results  than  others.  In  Section  III,  advantages  and 
disadvantages  of  each  method  are  given  and  a  "best"  method  is  chosen. 

n.  DISCUSSION  OF  METHODS 

In  this  section  a  detailed  discussion  of  the  four  indirect  methods  is 

given. 

A.  Prony's  Exponential  Approximation 

From  the  Bet  D,  four  equally  spaced  (with  respect  to  time)  points, 
(ti.yi),  (W 2),  (tj,y3),  (t*,y4)  are  chosen.  Let 

Ai  ~  A*"ir 

■  6 1  +  0ji  a:  -  0 j  -  flji. 

Then  0,efljt  Bin  flst  ■  Ajeait  +  A2eajt.  Prony's  theory  states  that  ea‘  and 
satisfy  the  equation, 

r2  +  Ctr  +  C3  -  0  (4) 

when  (4)  is  the  characteristic  equation  of  the  assumed  difference  equations 
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I 


-1-  C,y2  +  y,  «  0 

(  -v..  -  C.y,  +  V4  «=■  I)  .  (5) 

Thus,  From  1  ?!  and  (V. ,  u,  1  ■  a  a2  are  determined.  Next,  the  system 

*1  t  iift* 

> ,  *  A,o  ■  +  A2e  * 


*  A,r 


>a’tj  4  \..eb,t 


\  (>) 


js  .--(lived  to  hnd  At  and  A2.  Since  <»,,  f»2,  ant^  are  Riven  in  terms  of  a,,  a2,  A(, 
i.n  i  A-,,  i,,,  >  ,  .uhI  m-)  caii  ht  found. 

h.  vmnr  aeries  —  i.east  Squares  Polynomial 

‘•!i>th  an  d  by  the  imu'v-ds  01  the  scatter  diagram  of  the  data,  a 
tlucd  ;le>tri  f>  polynomial  P(ti‘  -  a„  +  att^  a2t*  +  a2ts  was  least  squares  fitted 
to  the.  dnta.  Thus,  a  .,  a,,  a2,  and  a3  are  determined.  Next,  y(t)  «  0te6it  sin  (i3t 
is  expanded  m  a  Taylor  series' about  t  *  0  and  the  series  is  truncated  after  the 
first  four  'erms.  From  the  least  squares  polynomial  and  the  truncated  Taylor 
series,  coefficients  of  oqtud  powers  of  t  arc  equated  yielding: 

n„  *»  0 

:,i  * 

Jlj  B  02020| 


1 1,.': 


1  ■,  '  r 


7  1  4  2020301  " 


i,,  ire  km  un,  f  T )  can  tie  solved  to  obtain  0\,  02*  and  ©3 : 

-an 


(7) 


f?2  « 


Ri 


'h 


[xul  -  fia.) 
v  nj 


Jh. 

f'3 


(R) 


so 


Unfortunately,  considerable  error  may  be  introduced  by  the  least  squares 
fit  of  the  polynomial  and  by  the  truncation  of  the  Taylor  series  after  four  terms. 


To  improve  our  estimate  of  0,  the  following  technique  is  applied.  From  the 
set  D,  the  set  D*CD  is  selected  such  that  (t*,y*)  eD*  and  (t,y)  cD  and 
t  =  t*  —  y*  2  y.  The  Taylor  series  —  least  squares  polynomial  method  is 
applied  to  the  set  D*  and  we  obtain 


Now  select  D^CD  such  that  (t,,, ,  t^ )  eD*  and  (t, y)  eD  and  t  -  t^  —  y,„  2  y. 
Again  apply  the  Taylor  series  —  least  squares  polynomial  method  and  obtain 


Actually,  what  we  want  to  do  now  is  to  "scan  the  interval  between  *0 


and  to  get  the  beat  estimate  for  0. 


Ideal  Picture 


K  i-»  eon*?  pre.issign*  d  fonstant  that  determines  step  length. 
Lot 


For  notations  purpose,  let 

®l  "  j V  ’  1  "  1,2,3  ‘  1  "  0,1,2,,,,K 


Nito  :  j  is  the  number  ol'  the  step  in  the  "scanning"  procedure. 

Lot 
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W/mw  C\(  a\  {a  Krtdhni  ncr  uHfh  1  ■  0 

—  ''yf  j - ** - *  "  - ■»  —  * 

Let  j+1»  =  ^  +  AandQ^  +  10^  is  computed.  IfQ^+1oj  >  Q^.  sct 

/l\m 

0  =  +  A^~l  beginning  with  m  =  1. 

Again  we  compute  Q^+1©j  ■  Q(j  +  lfi)  >  ^j6)’  we  ^  m  *  2  and 

repeat  the  above  (If  necessary  for  m  »  3,4,5  ).  If  <  Q(®1  lor 

m  ■  1,2, 3, 4,  we  set  ^  +  10  «*  0^  +  A  and  repeat  the  original  procedure.  Then 


we  use  m  ■  5;  even 


though  we  may  not  have  Q^+1oj  <  .  we  put 


0  »  ^0  +  A  and  repeat  the  original  procedure. 

It  is  desirable  that  0  and  the  residuals  be  printed  out  in  each  step  of  the 
search  so  as  to  gain  some  Insight  of  the  relationship  between  0  and  the  residuals. 


solve 


In  a  complete  description  of  the  first  Iteration,  wa  are  trying  to 


fyh  -  0jeOjt  sin  03 tj  *  0 


0  -I  0 


To  this  end,  an  initial  estimate 


0®  “I  0® 2 
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>/,t 

is  made.  Next  </,e  sin  (03t)  is  expanded  in  a  Taylor  senes  about . h  md  or  iv 
the  flr<»  two  Urn.  *»f  the  series  are  used.  The  truncated  Taylor  scries  replaces 
r,e'';l  sin  flit  in  .  Then,  the  partial  derivatives  with  respect  to  0(,  fl2,  and 
ot  Q<>>)  arc  ronuitpii  tnd  Het  equal  to  zero.  This  vcf  ulU  in  a  system  of 
equations  that  i-<  linear  ir.  ^  *  1.2,. a.  Thus,  if  the  coefficient  n,.iliix 

<>f  the  system  is  invertible,  .  c  can  solve  tor 


There  are  cases  m  which  the  coefficient  matrix  may  not  be  invertible.  Tins  Is 
certainly  true  it  the  parameter  surface  is  flat.  Thus,  a  singular  coefficient 
matrix,  in  this  case,  is  indicative  of  a  non-unique  solution.  In  cases  in  which 
a  unique  solution  dues  exist,  the  addition  of  second  partials  in  computing  the 
correction,  AO  helps  to  insure  the  non-singularity  of  the  coefficient  matrix. 

Effective's1,  we  have  a  new  estimate  for  0.  (This  is  where  the  differen¬ 
tial  correction  method,  alias  Newton's  Method,  begins  a  new  Iteration. ) 

Next,  we  optimize  the  magnitude  of  the  correction  A0.  The  explosion 
yj  +  vAO  n  s.  v  s  1  Is  considered. 

The  term  Q  Is  evaluated  at  ,,0,  <fl  +~ A0,  and  J)  +  A0 .  A  parabola  Is 
then  passed  through  these  three  points  and  the  minimum  of  the  parabola  is  cal¬ 
culated  in  terms  of  v.  Then  Q(„0  +  vA0)  Is  computed.  The0  associated  with 
the  min  |qu0)  ,  A  ~  AffJ ,  q(o0  +  A0) ,  Q(o0  +  vA0)  j  i8  chosen  and  called 
,0.  If  Qf.f-i  <  QUO)  the*  Initial  estimate  of  0,  „0  is  replaced  by  ,0  and  the 
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entire  procedure  is  repeated.  If  this  is  not  the  caBe,  the  domain  of  v  is  dimin¬ 
ished  ^usually  by^  and  the  optimization  of  the  magnitude  of  the  correction  Ad 
is  repeated  the  required  number  of  timeB  to  produce  Q(,0>  <  Q(0fl) ,  or  the 
domain  of  v  is  sufficiently  small  and  we  terminate  the  procedure. 

NOTE:  The  reason  that  the  check  of  the  min 

Q (ifi  +  vAG)J  is  considered  in  that  the  minimum  of  the  parabola  does  not  neces¬ 
sarily  occur  at  that  value  of  e  that  will  produce  the  minimum  Q. 

D.  The  Method  of  Steepest  Descent  —  Optimum  Gradient 
Again  we  consider  the  expression 


i  /  1  \ 

jQ'i';).Q^r,0  +  yA9j,Qf,f>  +  AG), 


Q(0)  ■  2j  yK  -  ®i  e  sin  (ejt. 

h-1  n  \  / 


An  initial  estimate  is  made;  call  it  qfl.  The  gradient,  VQ,  of  Q  is  computed  at 


0G.  Since  the  gradient  pointB  in  the  direction  of  maximum  Increase  of  Cj,  the 


negative  of  the  gradient  will  point  In  the  direction  of  greatest  decrease  of  the 

function.  Now  the  gradient  is  normalized  by  dividing  each  component  of  the 

gradient  by  the  maximum  of  the  absolute  values  of  the  components. 

We  next  optimize  the  step-length  in  the  direction  of  steepest  descent  by 

considering  the  function  r 

g(a)  ■  Q  jG  - 

we  find  that  value  of  a  that  will  make  Q  a  minimum. 

Now 

g'(a)  m  -VQ^  •  VQ  jO  -  «VQ 
By  setting  g'  ( a)  ■  0  —  and  VQ  ^  "  «VQ  are  orthogonal  to  each 

ot)ier  for  the  value  of  a  that  makes  Q  a  minimum, 
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V<  now  the  function  values  of  g  beginning  at  and  continuing 

'  '  '  1  '  '  ’  '•  '•  1  '• H  1  ''”n  the  s]opo  of  *  front  negative  io  non-negative 

"r  '  ,Mi'  W  ‘  at'x'i4 1  wh,chr V(>r  r(,n,<?3  Sup pore  wp  have  the  change  of 

'•  '  t‘r  r  in  M:iK,afU,ic  of^i  at  N.  Then  rtof'no 

H  •  N 


l,,‘  1  1 g>  ""  >'•  'h.g’fbl  I  we  will  pass  a  cubic  through 

'  mr  ,h<  vnlu"  °*  «  that  produces  a  minimum  in  t hr 
'  ""  t,  ir:ihvl  1  im‘Ti"il- -di.",  m  the  Newton- Gauss  program,  a  #<heck 

i  •  ti-  see  if  .v.  arc  derroatiing  the  magnitude  of  Q. 
the  1 ”  o  i>  determined,  we  put 

"ii'.fl  ornpnte  ; 

<;(,»)  »nd  ofMj. 

'  '»  ««  »*»*  over  four  Itentiou,  the  procett. 

f-  M«'th'»f  ot_Di fferontiai  Correction 

V  =  f  (t,  0 ! ,  6  9) ,  ^  i  o) 

•".'I'P  tmv  fortiiuia  to  ,  <’  n  good  fit  to  the  data  *  1....N).  The 

■  t<i.i l.-!  i rc  give  n  fiy 


8  f> 


J  Rj  =  ffti.tii.tfji#])  "  Yj 

R2  =  f ^ tj, 6 j, 0 j)  -  Yj 

\  -  -  Yh  (13) 

where  y^h  =  1,2,  ...n)  are  the  given  (observed)  values  from  the  original  data. 

Let  o0i,  n02,  063  be  an  initial  guess.  Now  we  need  to  correct  this  guess  by 
some  incremental  amount,  say  a,  p,  y  such  that 

0|  =  o®i  +  a 

02  “  <fil  r  P 

03  “  063  +  Y  (14) 

will  yield  a  better  fit  to  our  data. 

If  we  substitute  the  values  of  (14)  into  the  residuals  (13)  and  transpose 


the  y, ,  we  have  the  following; 
n 


^  +  yh  ■  001  +  «.  002  +  *&  0®3  +  vj 


Expanding  the  right-hand  side  by  Taylor's  theorem  we  get 


\  +  yh  "  f(v°e‘,(|fl*’°0s)+  a(Wr)o + 


/  MA  /0*h\ 

>02/o  \®0*/O 


+  higher  order  terms  in  a,  (3,  y  , 

fa f\  gf 

where  (rr—  1  *  the  value  of  the  partial  derivative  —  at 
Vy/o  9y 

t  *  t^,  0i=  j0i,  Oj  =  J0J,  and  0j  =  q0 j 


Our  first  approximation  is  obtained  from 

Y»  =  f(t,  001,002,  003) 
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so  that  we  have 


f(th,,0 ofls)  •*  .  '  (i 8) 

This  can  also  h  -  pot  into  ccuaMnn  (16),  Now  let 


Ignovfng  th»»  higher  order  terms  the  residuals  now  have  the  lorm 


which  are  liner  r  in  r«,  ■  .  Therefore,  we  may  determine  the  correct. ora  by  the 

k. 

method  of  least  squares. 

III.  RESULTS  AND  CONCLUSIONS 

These  five  methods  of  estimating  6  were  used  on  data  taken  from  actual 
test  firings  in  which  wc  hod  four  test  modes  (a  standard  and  three  modifications) . 
Plots  of  typical  data  are  shown  in  Figures  1  through  4.  The  results  of  all  five 
methods  are  shown  in  Tables  1  through  IV. 

Some  rd  vantages  and  disadvantages  of  each  method  are  given  in  Table  V. 

1 ;  i 

It  turns  out  that  the  "beat"  method  to  use  depends  on  the  tools  that  one  has  on 
hand.  For  example,  if  one  has  to  estimate  the  parameters  by  use  of  only  paper 
and  pencil,  he  naturally  would  chose  Prony's  method.  If  one  had  access  to  a 
small  computer,  he  might  chose  the  Taylor-least  squares  approach.  Of  the 
five  methods  discussed  in  this  paper,  the  method  of  steepest  descent,  and  the 
meth  xi  of  modified  Newton-Gauss  are  the  most  accurate  in  terms  of  the  smallest 
residuals. 
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TABLE  I.  STANDARD  DATA 


*2 

Residual 

Prony's 

6.9 

-1.14 

,  7.  :t 

374 

Taylor's 

modified 

6.4 

-2.  8 

7.6 

*>4 

' 

Differential 

- 

Did  not  "onvnrj't 

correction 

Steepest : 

6.5 

-2.  5 

7.9 

ill 

descent 

Modified 

Newton- 

Gauss 

6.5 

-2.5 

7.  9 

91 

TABLE  IT.  MOD  1  DATA 


f>2 

*3 

Residual 

Prony's 

1 

-1,24 

10.  3 

171 

Taylor's 

modified 

1 

-3.7 

10.  0 

88 

1  Differential , 

1 

correction 

2.07 

-1.85 

12.3 

68 

Steepest 

descent 

2.07 

-1. 85 

12.3 

68 

Modified 

Newton- 

Gauss 

2.09 

-1.  84 

12.3 

08 

R9 


TABLE  m.  MOD  2  DATA 


3. 

*2 

Residual 

Prony's 

5.3 

Jft.  7 

10.  46 

407 

Taylor's 

modified 

4. 14 

-2.5 

8.  92 

177 

Differential 

correction 

Did 

not  com 

’ergo 

Steep  set 
descent 

4.7 

-1.  5 

9.4 

102 

Modified 

Newton- 

Gauss 

4.8 

-1.5 

9.4 

101 

TABLE  IV.  MOD  3  DATA 


0*  • 

0J 

Residual 

Prony's 

8.6 

-0.6 

7.2 

820 

Taylor's 

modified 

8.0 

-2.1 

7.7 

328 

Differential 

Did  not  converge 

correction 

Steepest 

descent 

8.5 

-1.4 

7.3 

153 

Relative  min  due  to  guess 


Modified 

Newton- 

Gauss 
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TABLE  V.  ADVANTAGES  AND  DISADVANTAGES 


Sensitive 

to 

Guess 

— 

ou 

ou 

V/N 

v/N 

Relative 

Accuracy 

Converges  to 
Relative  Minimum 

N/A 

yes 

no 

yes 

yes 

■ 

Easily  Adapted 
to  Small 
Computer 

8  8  p  ■)  o 

>,  t*  a  a  a 

Easily 

Hand-fit 

yes 

no 

no 

no 

no 

Prony's 

Taylor's  -  least 
squares 

Differential-correction 

Steepest  descent 

Modified  Newton- 
Gauss 

91 


STANDARD  VEHICLE 


FIGURE  3.  MODIFICATION 
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A  METHOD  OF  IMPROVING  THE  ESTIMATION  ■  y  V  *  I 


John  tiurland 
University  of  Wisconsin 


J.  S.  Mehts 
Temple  University 


1:  Introduction. 


Consider  a  situation  where  an  experimenter  has  an  unbiased  estimator 

2 

of  the  population  variance  from  a  normal  distribution.  Let  us  further 
„  2 

suppose  that  another  independent  unbiased  estimator  of  the  population  variance 
is  also  available.  The  two  estimators  may  have  been  based  or.  sample:  ta».vr.  ot 
different  times  and  places.  Bacauaa  of  the  circunstances  it  may  be  kncv-r. 

that  s^  estimates  while  may  be  estimating  <>2  e*>.  This  shift  in  the 

variance,  if  anyfmay  have  taken  place  because  of  the  time  lapse  between  obtain¬ 
ing  independent  samples  or  it  could  b#  dua  to  the  shift  in  pieces.  It  is  <sb- 
2  2 

vious  that  if  cJ»Oj  then  the  two  estimators  can  be  pooled  to  obtain  a  "batter 

2  2  2  *2 

•Btimator”  of  o^.  ;0n  tho  othir  hand,  if  then  one  cj  iy 

2  2 

slope.  A  preliminary  teat  of  the  hypothesis  Ho:  can  be  carried  out  by 

utilizing  the  FCss^/e^  statistic  <yid  if  .'the  hypothesis  is  rejected  then  one 

2  2  2 
uses  Sj,  to  estimate  0^  otherwise  a  poqJLed  estimator  of  c^  is  obtainsd  by 

2  2 

pooling  s^  and  »2  appropriately.  The  estimator  designated  here  as  U  has 

been  obtained  by  following  this  approach,  which  is  similar  to  the  approach  used 
\  • 
bv  Bancroft  Cl],  ' 

Another  method  of  estimating  a*  is  to  usa  weights  which  are  continuous 

functions  of  F.  The  estimator  S  described  below  is  constructed  in  this  manner 

and  turns  out  to  be  more  effective  than  the  estimator  U, 

Let  x^|,  *12  *  *  •  •  *  ^  end  *22  *  *2N 

be  independent  samples  from  normal  populations  with  unknown  variances  0}  and 
O1  respectively ,  and  let  k  *  aJ/a*  .  It  is  required  to  aatimate  o1  .  Define 


as  usual 


N  N 

•l  *  “SPT  jfil  <xij“  xl)2  *  *2  '  TIT"  jJ1(x2j"x2)J 


I 


We  consider  an  estimator  T  of  weighted  sums  of  jr i 

cf  - 1” 


where  k>  <3  sctc.  ivr.cxw.  -  f  *\  ’.'..v  jvasnr.  for  considering  T  <*  an 
estirator  of  j‘  is  that  f.‘  is  an  estimator  based  only  on  the  first  sairole 


is  ij\  estimator  based  on  combining  the  two  samples  when  0*  =  0*  .  As 


an 


estimator  of  o:  we  consider  a  subclass  S  of  T  which  reduces  to  the  form, 

i  i 


2  °1 


V  +a3f 


.2 


-1 


(2) 


where  the  constants  of  0^  and  are  given  below.  This  estimator  has  also 

been  considered  elsewhere  (Mehta  and  Gurland  [6]  but  we  include  it  here  for 

*•  *  * 

comparison.  « 

The  estimator  b  mentioned  above  is  also  a  subclass  of  T  with  the 

w*;.?r  :  rincti  ’  v1  given  by 

0  for  1/F  <  F  <  F  . 

i  o  o  , . . 

j  -  •!  ( 2 ) 

^1  otherwise 


9fl 


The  const  in t  \  \a  determined  by  the  i-  v*0  ef  -  ' gni ficanc/-  A  the 
o 

preliniiidry  test  fcr  equality  of  variances. 

The  behaviour  of  S  and  U  will  be  investigated  in  regard  to  ex¬ 
pected  mean  square  and  relative  bias. 

2.  The  estimator  U 

The  estimator  U  is  of  the  tom  T  wj.tr,  the  -we ;.;;rit  functior  dir.ntd  ty 
(3)  above.  It  is  based  on  the  randen  outcome  of  a  preliminary  test  of  vne^her 
O1  a  a*  .  If  the  preliminary  test  rejects  equality  of  variances  then  is 
employed  as  the  estimator;  but  if  it  does  not  reject  equality  of  variances 
then  the  average  of  s£  and  is  used  as  the  estimator  of  0*.  Thiu  estimator 

is  similar  to  one  used  by  Bancroft  [1 j  except  that  we  use  a  two-sided  test  for 
equality  of  variances  whereas  he  uses  a  one-sided  test. 

First  we  consider  the  expected  mean  square  of  U  and  compare  it  wi-h""'" 

2  l  1  2 

that  of  which  utilizes  only  the  \first  sample.  Since  s£  is  unbiased  w«  de¬ 
fine  the  efficiency  of  U  as 

Var  s* 

Eff  U  »  - ~  (4) 

E(U-0*)2 

Subsequently  we  shall  also  consider  the  relative  bias  of  U  given  by 


E(U)  -  dj 

of 


(5) 


In  order  to  obtain  expressions  for  the  efficiency  and  bias  of  'J 


we  need  to  evaluate  the  first  two  moments  of  U 


where 


bias  U  =  ECo)  -  0‘ 


and  E(U),  E(lr)  are  given  above. 


3.  Sore  computed  values  of  Efficiency  and  Relative  Bias  of  U. 

In  Tables  1-5  are  giver,  calculated  values  of  the  efficicnc”  cf  u,  fer 
.cample  sizes  3,5. 7, y, 11,  f:r  the  rnr.uo  0.1  <_  IC.C,  and  for  valuer  cf  the 
constant  F  which  cor  re  sc  or.d  to  1'.  ,0  j.lC'l.IOl  and  501.  level  r.  cf  si  grificancc  . 

V  ' 

Examination  of  these  tables  reveals  tnat  there  is  a  gain  of  efficiency  for 
some  values  of  k  ar.d  a  loss  for  other  values  of  k.  Furthermore  the  extent  of 
these  gains  or  losses  depends  on  sample  size.  For  k  <_  I  there  is  generally 
a  gain  in  efficiency  but  the  magnitude  of  this  gain  decreases  with  increase  ir. 
sample  size.  As  3  matter  cf  fact  for  small  values  of  .  e.g.  a rtunc  '.1. 

there  is  even  a  }•  ss  cf  efficiency  as  N  becomes  larger. 

As  far  as  values  of  k  >  1  are  concerned  there  in.  without  exception,  a 

j 

general  loss  of  efficiency  as  manifested  for  all  the  sample  sizes  considered. 
The  relationship  of  this  loss  with  the  values  of  k  and  J?  is  more  complicated 
than  for  the  case  k  <  1  considered  above.  For  some  values  of  k  >1,  e.g: 

k  =  2,  the  loss  in  efficiency  becomes  mere  pronounced  as  ”  increases  while  for 

* 

other  values  of  k  >  1,  e.g.  k  *  10,  it  becomes  less  pronounced. 

For  all  values  of  k  and  II  considered  the  relation  of  efficiency  to  tr.c 


value  of  the  constant  FQ  follows  a  definite  pattern.  Whatever  be  the  sample 
size,  the  efficiency  for  values  of  k  >  1  increases  with  decrease  of  Fc  (or 
equivalently  with  increase  of  level  of  significance  of  the  preliminary  test), 
On  the  other  hand  for  k  <  1  the  trend  is  reversed,  that  is  to  sav.  the 
efficiency  decreases  with  increase  of  the  level  of  significance  of  the 


preliminary  test, 


Best  Available  Cop'; 


0:. 


estimator  o:  this  c’aus  with  a  low  level  cf  uignif  -Cj:.co  of  the  r.re 
Tost  involves.  however,  when  thin  xevo_  is  low  the  relative  bias  1 
Ir.  Table  1.  is  presentee  the  relative  bias  of  U  for  sample  sites  1 


„ ,7,0  anc 


11.  7Y,e  value  of  Fq  ,  for  each  sample  site,  corresponds  to  a  201  lov-i  of 
sipnifiuar.ee  of  tr.c  preliminary  teat  of  the  hypothesis  HQ:  k  -  1.  Other 
levels  of  significance  for  too  preliminary  test  are  possible,  of  course,  arc 
a  matter  of  fact  ir.  'i'a.lec  1-0  wo  hr ve  already  discussed  tr.c  erficiuncv  orb 
for  values  cf  Fq  ccrro sconcing  to  levc-xs  !.>,;/*, and  50'u.  hew  for  itve 
greater  then  2Cv  the  relative  bias  will  bo  smaller  but  at  the  same  time  the 
gain  in  efficiency  will  also  be-  snax-cr.  On  the  other  name  for  level;,  .cos 
than  2 Cl  t'r.a  efficiency  will  be  higher  but  the  relative  bias  will  aloo  be 
higher.  At  this  particular  level,  namely,  20%,  the  relative  bias  is  reason... 
well  controlled  for  0.1  <  k  <  1.0  and  at  the  same  tiqe  there  arej  gains  cf 

i 

efficiency,  at  least  in. a  subset  of  this  range.  On  reforinc  to  T’able  1C  we 

*  '4 

note  that  the  maximum  relative  bias  in  this  range  of  k  for  sample  size 
I!  -  11  is  u .  ana  for  N  =  3  it  is  l**v. 

I 

4.  7ne  estimator  S 

In  the  general  estimator  T  defined  by  (2)  let  us  regard  ip  for  tne 
moment  as  a  constant,  and  minimise  the  expected  moan  square  error  of  7  wit:, 
rospuct  to  .•  .  Thu"  minimum  if;  reached  when  i 

..  _  N-3-2k(N-.)tk2(f;+l) 

w  *  - - - 

(N+l)-2k(N-l)tkZ(N+l) 


( 
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Tf  uo  nhd  t  tnj  *:  fAf»  if#  £n  thn  «Qtin>atnv»  T  _an/4  vw»  pi  ar<»  l 

which  is, of  course,  unknown,  by  c+dF,  where  c  and  d  art-  arbitrary  constants, 
we  obtain  the  estimator  S  given  by  (2),  where 

a  =  (N-l)(l-2c)  t  (N+l)c2  6X  =  (N+l>(l-*ci)-20.,-i)c 

a2  =  2{(N-l)d-(N+l)cd-l}  B2  s  2  d  { ( N- 1 )  *  (Mtl)c) 

u„  =  (!!+l)d2  S.  =  {H+l)d2 

o  3 

Tne  relevant  underlying  details  involved  in  obtaining  the  above 
estimator  S  are  outlined  in  the  paper  by  Mehta  and  Gurland  [4], 

Estimation  of  k  by  a  simple  function  such  as  c  +  dF  has  been 
applied  similarly  in  other  contexts  (cf.  [2], [3]).  The  constants  c  anc  c  .r.u?.t 
be  appropriately  chosen,  and  the  results  of  certain,  choices  will  appear  in 
Tables  £,  7,  8,  9,  11,  12  considered  below. 

As  in  the  case  of  U  we  require  the  first  two  moments  «of  S  in  oder  to 
evaluate  its  efficiency  and  relative  bias,  for  odd  values  of  the  sample  sice 
N  these  moments  can  ba  expressed  as  a  finite  series  of  integrals  which  can  be 
evaluated  by  reduction.  The  precise  form  of  these  moments  appears  in  the  work 
by  Mehta  and  Gurland  [6]  cited  above. 

5.  Some  ccrnuted  values  of  Efficiency  and  Relative  Bias  of  S 

In  examining  the  behaviour  of  the  estimator  S  we  employ  the  same 
criteria  of  efficiency  and  relative  bias  defined  above  as  in  (4),  (5),  for 
the  estimator  U.  The  behaviour  of  the  estimator  S  has  bean  considered 
previously  in  [6],  but  for  convenience  of  making  comparisons  with  the  estimator 
U  we  sketch  these  results  here  briefly.  In  table  6  the  efficiency  is  sh.Ara 
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when  values  o:  ■:->r.£ tants  c  and  d  have  been  selected  to  empiss '.zt  the 

range  a  j.  except  lor  very  small  sac-pie  sizes,  suet,  as  »  <  3, 

the  gain  in  efficiency  is  not  substantial.  In  fact,  a  loss  in  efficency 
begins  to  occur  tor  1-:  size  7  and  for  larger  values  of  V. 

in  Tan  l  *  ’’  •  "h  *•:'■' iciercy  .s  shewn  -.\>rresponding  to  constants  c 
and  d  which  emphasise  tne  range  0.1  <  k  <  1.0.  There  is  a  considerable 

gain  ir.  efficiency  for  tf..'-.  range,  especially  for  small  sample  size.  This 
gain,  hiwvi",  it  :■?.  ornt-T  fed  by  -  larze  Illative  bi.if . 

r.  I  able  ?•  tn.  r.'ficiencv  is  given  for  estimator  0  where  the  constants 
c  and  d  are  chosen  so  that  the  relative  bias  remains  numerically  below  10i 
for  the  range  0.1  <  k  <  10. C.  Toole  3  indicates  the  efficiency  that 
res  lilts  when  constants  c  and  d  are  chose  to  hold  the  relative  bias  numerically' 
below  Si.  In  this  case  the  gain  in  efficiency  is  slight,  especially  for 
larger  sample  sires. 

In  TcMes  jl  and  -1’  are  presented  the  relative  bias  of  the  estimators 
in  the  cl-rs  S  for  which  the  efficiency  has  been  discussed  in  Tables  3  af.rt  9. 

r'ro:'  T.i:  i  i  it  is  evioont  that  the  relative  bias  of  the  estimators 

.  ■■  .  tbr-r«  >  n:ch  smaller.  In  fact  for  N  «  11 'the  maximum  relative 
hi  i..  i  . .  ii;,.:  for  *  2  it  is  f-.8"0.  The  gain  in  efficiency,  however,  as 

in  .  ica*c.:  is  Table  3  is  ..slv  very  s light  for  the  range  C.l  <  k  <  1.0; 

:  -r  •  ...  ■  ril  .  sli;,st  loss  of  efficiency  far  the  range 

.  :■  *  ■  •  :  "rim  tst  point  of  view  of  efficiency  this  estimator  is  not 

sc  .:t tr  .  :  ve  ;  however,  it  is  of  interest  if  one  is  mainly  interested  ir. 

c  .s'  r  . .  ir the  bias. 


1Q4 


It  in  evirc: t  from  Tabic  ii  that  the  max mu-  r<.:.itive  i„  ;  :.-.e  w.'.pi 

r-;..  c  0.1  <  k  10  3 r.d  for  all  the  sample  sizes  considered  is  lab.  r.  is 
Jino  evident  thut  for  many  of  tnese  values  of  k  for  t he  sample  sizes 
considered  this  relative  bias  is  very  small.  This  control  of  relative  bias 
together  with  the  gain  in  efficiency  discussed  previously  would  indicate 
that  this  member  of  the  class  S  merits  consideration  as  an  estimator  of  o^. 

5.  Comparison  of  the  behaviour  of  estimators  U  and  S 

In  comparing  the  benaviour  cf  S  and  U  it  is  necessary  to  keep  ir.  ;.-.ir.d 

that  the  parameter  k  can  assume  values  greater  than  or  less  than  or.e.  It 
is  evident  that  we  can  find  members  of  the  class  S  which  in  most  of  the 
range  ,0.1  <_  k  <_  10.0  are  more  efficient  than  members  of  U  . .  For  example  or. 
comparing  the  estimator  S  in  Table  6  corresponding  to  N  =  9  and  the  estimator 

U  ir.  Table  4  corresponding  to  a  preliminary  test  at  a  20v  lpvel  wo  observe 

that  for  all  values  of  k  >  1  the  efficiencies  of  S  arc  very  much  higher  than 
those  of  u>  For  the  range  0.1  ^  k  <.1.0  the  efficiencies  of  ‘S  exceed  those 
of  U  except  for  values  of  k  in  the  subset  0.7  <_  k  <_  1.0  in  which  subset  the 

N 

efficiencies  of  U  are  only  slightly  greater  than  those  of  S.  The  comparative 
behaviour  of  3  and  U  for  other  sample  sizes  considered  follows  a  similar 
pattern.  Generally  speaking  therefore  in  the  whole  range  0.1  <  k  <  10. C  the 
estimator  f.  appears  preferable  as  far  as  efficiency  is  concerned. 

bet  us  now  consider  the  relative  bias  of  these  estimators.  Values  of  the 
relative  bias  of  S  and  U  are  given  in  Tables  11  and  10  respectively.  Cn 
comparing  these  biases  corresponding  to  sample  size  N  s  9,  for  example,  the 
relative  bias  of  S  in  the  range  1.0  <  k  £  10.0  is  very  much  iC3s  than  t:\at  of 
U  ,  while  for  k  in  the  range  0.1  k  £  1,0  the  relative  bias  of  S  remains  less 


.  an  r  h'.t  r>i  U  v.*r  ilo 
ran  that  of  U  cxcc.pt 


0  '  ..  ■  1  L’  -  .  •  c  <:n  ..vc  in  rites  ;,i« 

r.-l,,tlvc  .  v  •.  ■  i  or  . . :  :s  cr'-nz.  wit  r  a  prei  i-ir.ar;,  test 

■  ? 

at  '.'v'-  level  :v»  results  iniicute  t  -.it  it  we  ro-ard  efficiency  and  relative 
Idac.  •? .1 .”  ..  *. .  .  r  ..;■<•  1.  -  ■:  •  „•/. ..  ;r.a  es . Imator  d  is  prefer..:,  ie. 

:.c"  '  .  ..  •  T. i,  it.  .  1  .  ai-c  possible  : select..-.^ 

diffarvt  members  n;:  these  .r.artos  r-r  estimators;  however,  we  believe,  the 
basis  c.*  cr - -.r Inor,  :v: leered  boro  -a  the  r.ont  generous  towards  U  .  I.',  for 

example,'  we  consider  the  class  U  with  a  level  of  S0<1  for  the  preliminary 
test,  'it',  rviativo  bias  is  unmoved  but  the-  efficiencies,  with  very  slicht 

9  y<  'i  •> 

exceptions,  are  all  lose  than  those  of  Si  On  the  other  hand  if  we  consider 

') 

the  class  U  with  a  preliminary  test  at  a  low  level,  for  example  1%  the 
efficiencies  for  k  *.  1  fter.crallv  exceed  those  of  S:  however,  the 
disadvantages  cf  much  a  U  would  be  overwhelming  because  its  relative  bias 

is  prohibitive!”  lar.;*  . i r.c:  its  Losr-  of  efficiency  for  k  >  1  is  terrible. 

Other  estimators  S  beside:,  thor.  given  in  Table  b  night  also  have  been 
considered  for  the  comparison.  In  Tables  6  and  7,  for  example,  estimators 
are  •.  re r»  :it*..:  w.-.ich  emphasize  tr.c  ranr.es  i.G  <  k  <  1G.0  and  0.1  <_  k  <  1.0 
respectively.  On  the  ether  r.una  the  estimators  considered  in  Tables  9  and 
12  control  the  relative  bias  within  a  maximum  of  5%.  Tor  the  whole  range 
0.1  <  k  <  1C.0,  however,  the  estimator  considered  in  Table  8  is  worth 
re  corner,  cir.r  because  the  relative  bias  is  reasonably  well  controlled  and  there 
arc  noticeable  gains  of  efficiency. 
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Examples • 


(1)  The  f’Vi  1 1  nu  1  nr/  »vAmni*  hc“  to  e bs 1  p  £  sr;d  *j  is  a 

practical  situation.  For  this  we  have  drawn  a  sample  of  size  7  from  N ( 3 , 1 ) 
and  named  it  as  the  first  sample.  The  unbiased  estimate  of  o|  s  l  as  given 
by  s*  from  this  sample  is  s3  =  1,65?..  We  draw  another  sample  of  size  7  from 

N(5,0.36)  and  designate  it  as  the  second  sample.  The  unbiased  estimate  of 
=  0.36  as  given  by  s*  is  s^  =  0.4^8.  Consequently  F  =  cf/s*  ~  0*271 
and  the  hypothesis  that  k  *  1  is  rejected  at  the  20%  level  of  significance. 

Thus  U  =  1.653..  On  the  other  hand  we  obtain 

v- 

S  a  1,527  if  we  restrict  the  relative  bias  to  be  less  than  10%. 

=  1.623  if  we  restrict  the  relative  bias  to  be  less  than  5%. 

a  1.011  if  we  use  the  estimator  S  which  emphasizes  the  range  0.1  <  k  <  1.0. 

a  0.969  if  we  use  the  estimator  S  which  emphasizes  tha  range  1. 0  *  k  <  10.0. 

1 

In  all  the  cases  we  note  that  S  is  nearer  the  true,  value  ©f  unity  than  U. 

(2)  ?The  example  conaidered  here  differs", from  that  of  (1)  in  that  hem  the 

,  ,  ; 

value  of  the  ratio  k  is  greaterthan  1.  Suppose  now  we  have  a  sec6nd  sample 
also  of  size  7  from  N(4,»0.  The  unbiased  estimate  of  <j|  *  ‘♦is  *2  *  4i305,  ^h* 

value  of  F  >  is  now  2.603  which  is  not  significant  at  20%  level  of 

significance  and  consequently  the  null  hypothesis  that  k  >  1  it  not  rejected. 
Therefore  in  this  case  the  estimate  U  a  ,l-»3  +  4 ■ 303  ,  2>g79. 

On  the  other  hand  we  obtain 

S  3  1.660  if  we  restrict  the  relative  bias  to  be  less  than  10%. 

3  1.660  if  we  restrict  the  relative  bias  to  be  less  than  5%. 

3  1.881  if  we  use  the  estimator  S  which  emphasizes  the  range  0.1  <  k  <  1.0. 

=  1.663  if  we  use  the  estimator  S  which  emphasizes  the  range  1.0  <k  <  10.0. 

In  all  the  cases  we  note  that  the  estimator  S  is  nearer  the  true  veluo  than  U. 
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S i*. ior.ai  institutes  of  h»*rt**4; 


I  INTRODUCTION.  Eisenhart  *'  19»+7 >  distinguished  two  uses  of  analysis 
of  variance  which  he  designated  as  Type  I  and  Type  II.  Type  I  provides 
a  test  of  significance  of  the  difference  between  estimates  of  population 
means.  Type  II  provides  a  test  for  estimates  of  population  variances. 
Eisenhart 's  treatmer  t  covered  the  general  case  of  analysis  of  variance — 
but  involved  two  ijtpcrtant  types  of  restrictions,  first  the  "residual 
error"  was  assurod  homogeneous  with  zero  expected  value.  Second ,  all 
other  parameters  in  Type  I  were  assured  to  have  zero  variances ,  and  in 
Type  II  to  have  zero  means.  In  the  mixed  model,  parameters  oould  be  of 
either  form,  but,  individually,  where  the  means  are  not  assumed  zero  the 
variances  are  and  vice  versa. 

The  present  paper  is  limited  to  the  case  of  two  classes  (the 
bivariate  case)  but  removes  both  of  these  restrictions.  This  leads  to 
a  greatly  enlarged  variety  of  types  and  to  a  close  parallelism  with 
bivariate  correlation.  TVo  special  cases,  not  previously  treated,  are 
discovered  and  appropriate  formulae  derived. 

I!  MATHEMATICAL  MODEL.  Given 

x  =  a  +  S  +  Y  +  i  +  c+  .  .  . 

we  have 

E(x)  sa  +  I+  ytJt  e  +  .  .  . 

V(x)  *  V(a)  ♦  V($)  ♦  V(y)  ♦  V<6>  +  V(e)  +  .  .  . 

This  article  has  been  reproduced  photographically  from  the  author's 
manuscript. 
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:<  =  <  a‘*+0',+  'i 

ere  v:  in  Lincrc  ••  “he  parameters  either  have  zero  expected 

values  or  zero  variances,  bu r  not  Loth.  In  an  exactly  parallel  way  we 

say  express  a  sccr.:c.  v«e  :-jx  -  y  a? 


>,  -  C'.*  ♦  ~  >'  *  •, 


.  a*  +  3*  +  Y*  +  •  •  • 

it  wull  be  permissible  to  group  the 


t.'uta.v  and/or  the  vai -able  ijard.7#rters  in  x  and  y  ir.tc  some  lesser 
r.unher  -adequate  tor  the  purpose  in  hand.  If  in  particular,  x  and  y  are 
distributed  in  a  bivariate  normal ,  then  the  distribution  factors  into 
the  product  of  one  involving  the  means  only,  and  the  other  involving  the 
variances  and  -n 'variances.  For  our  purposes,  it  will  be  enough  to  ignore 
the  distribution  of  the  means,  and  to  express  x  and  y  as  the  sum  of  two 


X  =  T  +  C 


variables  cf  the  form 


\  s  T  +  7 

If  this  is  done,  the  variance- covariance  (dispersion)  matrix 


becomes 


pc  a 

x  y 


oo  a 

r*  y 


b2+a2 


where 


a2+a2 


a2  =  V(t);  =  V(0»  o|  *  V<n). 


xx  p  xji  u;  xt»  iicr^a  _xvg  j-  uieii  \x/  lb  Ui  lj  »c  iuiv 


and  (3)  be canes 


(4) 


(5) 


so  that  aorrparison  of  the  two  models  can  be  obtained  from  (2)  and  (3). 


Ill  CORRELATION'  MODEu.  Viewed  as  a  purely  ;/vithenatical  object,  the 
various  special  estimating  and  testing  problems  in  the  literature  and 
certain  simple  extensions  can  be  classified  on  the  basis  of  restrictions 
on  the  elements  of  matrix  (2)  as  follows: 


A.  Estimate  p,  a*  and  a*.  Test  p  *  0.  This  is  the  most 

x  y 

oarnncn  situation.  The  t-test  applies;  most  naturally  as  a  test  of  p. 

B.  Given  o*  and  c*,  test  p  *  0.  The  t-test  with  infinite 

x  y 

degrees  of  freedom,  the  normal  test,  far  p  =  0  applies. 

C.  Given  p  =  0,  test  o*  =  cA,  and  estimate  the  oomman 

x  y 

variance.  This  is  the  new  classic  case  of  estimating  and  testing 
equality  of  two  independent  variances. 

D.  Given  cA  5  oA  =  oi,  estimate  p  and  oi,  test  p  -  0.  This 

x  y  c  C 

example  was  treated  by  DeLury  (1938). 

E.  Test  p  s  0,  o*  *  oA.  This  is  the  compound  symmetry 

x  y 

problem  of  Mauchly  (1940). 

*  This  possibility  seems  to  have  previously  been  overlooked  CAnsccrrbe). 
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;  "  hut  l/rvi,,  te^t  3}  *  o.*  This 

i  rc  „m  .  .  ,  •.  Tire*'/  >,  !*•  :»8) .  Morgan  (19..-9)  shewed 

*-hat  !+•  '  ~r»\vr.T  rower  Than  the  test  in  G. 

5.  wj-i'v.-'v.  j  arxi  test  ai  -  c*  whatever  the  value  of  p. 

a  y 

"’his  test  was  indapt'ndfcntly  supplied  by  Pi  Tran  (1939)  and 
Morpan  (1939).  It  is  dasru'ibed  in  Gnedeeor  and  Cochran  (1967), 
Section  7.12. 

Now  suppose  x  >  the  sum  of  two  independent  random  variables 
t  and  c  end  simiianly  V  is  the  sirr  of  t  and  n,  as  given  in 
aquation  tl).  Then  p,  the  correlation  coefficient  of  X  end  Y 
is  pTeater  tnan  or  equal  to  aero  end  the  matrix  V  waa  shown  above 
to  be 


The  various  possible  tests  in  this  fornulation  are: 

li.  Test  o*  -  0.  Estimate  o*,  oj *  and  o*.  This  is  a 
test  of  the  correlation  of  X  and  Y,  expreaaod  in  the  language  of 
oorrrron  end  specific  variances. 

I.  Given  oj  and  o|,  test  a *  *  0.  This  is  ths 
components  of  verisnoe  analogue  of  B  above.  The  sas»  test 
applies . 
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T  fiuxr  r  H  anrl  t-*»cr  n*  r  n i  T)vic 

.  ■  l  -  J 

is  the  usual  case  of  repeated  measurements  on  constant  but  not 
necessarily  equal  standards;  more  generally,  the  test  for 
equality  of  two  independent  variances  as  in  C. 

K.  Given  cj  *  c\  ~  a*,  estimate  a2  end  a2.  Test 

o1  *  0.  This  is  DeLury's  problem  expressed  in  the  language  of 
variance  oanponents.  But  whereas  in  its  correlation  formulation 
|  the  problem  appears  to  arise  infrequently,  in  its  components  of 

variance  formulation  it  is  Eisenhart's  Model  II  for  Analysis  of 
Variance. 

L.  Teat  o2  *  0,  o]  *  a*.  This  is  Mauchly's  problem 
expressed  in  the  language  of  variance  oemponents .  The  oanponents 
of  variation  interpretation  of  this  test  would  be  applicable 

wherever  (a)  observations  occur  in  pairs,  (b )  the  variance  of  the 

l  ■ 

|  first  member  of  each  pair  is  to  be  ccrrpared  with  the  variance  of 

[’  the  seoond  member.  Thus  in  a  paired  comparison  experiment,  the 

residuals  of  the  first  members  of  each  pair  oould  be  compared  with 

I- 

|  the  seoond  as  a  test  of  the  mathematical  model  underlying  the 

design. 

M.  Test  o2  *  0.  This  tests  the  legitimacy  of  treating 
the  precision  of  one  of  two  instruments,  standards,  or  techniques 
as  subject  to  no  (negligible)  error.  The  test  is  supplied  in 

Maloney  and  Rastogi. 

N.  Given  o*  Jcrvown,  test  c2  *  oj. 
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COgl ,  ll'JtTn  w;u«  u»e 


-  u  J.  ••  ry 

Tr.f*  1  ix.jllr.ood  criterion  is 


~jHUi  -  7;;--n>o* 

X:  V2  *""()^  -  Y>*/2]*  -  nV ’ 


(23) 


for  large  n,  -2  log  X  is  . hi-squave  r,v.  with  one  d.f.  (see 

Wilx*,  M63,  d.apt  r,-  13  ■ .  Therefore,  we  reject  the  hypothesis  HQ 

ir  log  a  -  \?  vr,  ;•  :  ir.  upper  .OC  percentile  point  of  chi- 

uquare  random  (/ariaoie  with  one  degree  of  freedom 

portion  (23)  or.  he  written  in  a  form  which  will  be  useful 

tvei'jw  and  is .  .tnasrjnic  as  well.  Since  Sx  Sy  is  the  geometric  mean 

of  S2  and  S2  and  i  (S2  +  S2)  is  their  arithmetic  mean,  equation  (23) 
x  y  *  x  y 

become : 


,  (GM)2  -  eV 

A«/n  *  - 

(AM)2  -  eV 

writing  GM  for  geometric  mean,  AM  for  arithmetic  mean,  and 

e  -  n/ (n-i) . 

Returning  to  equation  (23)  if,  in  particular  o2  *  0,  the 
likelihood  ratio  becomes 


(24) 


,2/n 


S2  S2 

_J<_X 


(GM)2 


4F 


(AM)2  (1  ♦  F)2 


distributed 

(n-1,  n-1)  d.f. ,  since,  when  o2  *  0,  X  and  Y  are  independent  r.v.'s. 


(25 


i  — 

where  F  *  S2  /  S  is  distributed  as  Snedacor'e  F  r.v.  with 
x  y 
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7b"  tent  bao*--J  u.  '  in  to  reject  ii  if  0  *  *  <  ...  J> 

equivalent  to  C  <  F  <  k  l  or  k  2  <  F;  wner«t  k  <  ...  At,  T  is  always 
taken  to  be  greater  tJian  1,  the  xule  beccnet  rt:je" t  if  F  >  k* 

at  the  chosen  probability  level,  i.e. ,  our  test  reauces  to  the 
ordinary  f  test  when  it  is  known  that  the  population  variance  is 
zero. 

Comparison  of  equations  (24)  and  (25)  exhibits  the  effect  on 
the  test  of  the  existence  and  magnitude  of  population  variance. 
Equation  (24)  is 


/n  <GM)J 

_ 

(AM)2 

(GM)2 

-  &2ak 

A  m  • 

(AM)2 

(GM)2  * 

(AM)2 

-  e2o* 

(GM)2 

(AM)2 

(GM)2 

-  e2®* 

(AM)2 

(AM)2  ' 

(AM)2 

(GM)2 

-  e2o* 

(GM)2 

4F 

<  — . . 

(1  +  F)1  (using  25) 

since  AM  >  GM  for  any  set  of  positive  numbers.  It  follows  that,  if 
a  standard  F  test  is  applied  to  the  variance  estimates  for  the  two 
instruments  or  procedures  as  if  the  effect  of  population  variance 
were  zero  (equation  (25))  the  test  will  sometimes  accept  when  the 
correct  test  (equation  (23))  might  reject.  Conversely,  when  equation  , 
(25)  is  appropriate,  discrimination  will  be  sharper  than  in  a  test, 
situation  where  population  variance  is  present  so  that  equation  (23) 
must  be  used.  In  addition  equation  (23)  can  be  used  to  gain  insight 
into  the  benefit  to  be  derived,  hence  into  the  care  and  expense  which 
is  justified,  when  the  population  variance  is  reduced;  whether  by 
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CORRELATION 

COMPONENTS  OF  VARIANCE 

Case  Assumption  Test 

Case  Assumption  Test 

A  0=0 

H  a2  =  0 

B  a2,  a2  P  8  0 

I  crj,  q*  o2  =  0 

C  0=0  c£  =  Oy 

J  a2  =  O'  oj  =  c2 

D  cx  a  cy  p  =  0 

X  cr{  =  o2  o2  *  0 

E  p  «  0;  o*  «  o* 

L  o2  *  0;  a\  *  a\ 

M  o|  =  0 

F  o  $  2 (known)  o2  =  Oy 

N  o2  $  OOcncwn)  aj  =  o2 

G  c2  =  c2 

x  y 

0  a\  -■  a2 

Test  of  significance  for  variance-covariance  parameters  of  a  bivariate 
relation  according  to  all  possible  non-trivial  parameter  restrictions .  For 
all  correlational  models,  the  corresponding  components  of  variance  model 
yields  the  same  test  (not  the  same  estimates)  as  its  correlational  analogue. 
No  correlational  model  exists  corresponding  to  item  M. 
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COMPUTERIZED  QUALITY  CONTROL  Ai'  APFl  ILL 
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Oskar  M.  Essenvanger 
U.  S.  Army  Missile  Command 
Redstone  Arsenal,  Alabama 


ABSTRACT.  Any  observational  program,  oven  if  carried  out  with  the 
best  available  insv.rumentation  and  carefulness  to  avoid  instrumental 
error,  mav  contain  erroneous  data  intr-i’uoud  by  pro jurat I  -r.  ar.d  trans¬ 
mission  of  data.  Thus  a  good  concept  of  quality  assurance  must  precede 
any  data  analysis  to  avoid  distortion  and  bias  of  results  by  erroneous 
records. 

Three  groups  of  analytical  methods  of  quality  assurance  are 
discussed,  inconsistencies,  interrelationship  of  data,  and  frequency 
distributions.  These  methods  have  been  developed  at  the  Army  Missile 
Command  for  screening  radiosonde  data  by  high  speed  computers.  The 
goal  is  flagging  of  erroneous  or  suspicious  records  that  these  may  be 
corrected. 

Checking  procedures  include  tests  for  trivial  errors  such  as 
duplication,  wrong  sequence,  missing  data,  special  checks  on  identifi¬ 
cation  numbers,  etc.  Other  procedures  utilise  data  interrelationships, 
in  this  specie!  case  the  vertical  structure  of  the  atmosphere.  Further 
checks  employ  screening  of  maxima  and  minima  by  exceedance  criteria 
derived  from  the  frequency  distribution.  The  Weibull  distribution  has 
proven  especially  useful  in  this  last  phase  of  the  checking  procedure. 
Some  pitfalls  and  limitations  in  the  utilisation  of  evaluation  criteria 
are  discussed. 


The  remainder  of  this  article  has  been  reproduced  photographically 
from  the  author'B  manuscript. 
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itnaojuccioi. 


is.  iJ  a  b'.r*-!  li',  ,4v.v.jf.teJ  fact  that  ail  *nw  data  irom  sample  surveys 
uni  e:'ijtriuenr.0vf“-.*,iin  i-^.irs.  Even  if  an  observational  program  has  been 
carefully  prepared  anJ  id  carried  out  with  the  best  available  instru- 
-'•cn  tut  ion  which  tic  p.u?,r.-.r.  .  ..  *  veajo;uioly  «itord,  some  errors  are  always 
;  r.'som  ,  Th  jv  may  ’:<■  c..  i:  .4  by  io.s  ;r’unonta  I  d^ficiencioa  or  inaccuracies 
'  by  uaqu.'.  i.i*  fill  .vat  tar  also  be  introduced  by  preparation  or 

trancmiv.  ion. 

Arclyr.iu  of  obs orvavicnnl  d.ua  wan  be  no  better  than  the  qualify  of 
fV>  available  data.  Thus  a  careful  attention  to  quality  assurance  of  the 
data  must  precede  anv  data  analysis.  Tills  vitsl  part  of  any  investigation 
should  b«*.  a  r  concern  to  all  investigators.  Its  main  purpose  is  to 
avoid  distortion  of  the  analysis  resulting  from  erroneous  observations . 
This  goal  will  determine  the  magnitude  of  the  effort  to  be  put  into  a 
quality  control  program  and  will  influence  Che  methods  selected  for 
quality  assurance.  Some  ~esults  can  be  evaluated  for  soundness  by 
qualified  professionals  and  then  a  quality  check  could  be  omitted.  The 
complexity  of  the  atmosphere  or  the  amount  of  the  end  product  (such  as 
tables  of  matrices  or  computer  produced  maps,  etc.)  made  it  virtually 
impossible  in  our  case  to  Judge  correctness  of  the  results  afterwards. 

Of  course,  one  cannot  make  good  data  out  of  bad  records,  but  a  so- 
called  "editing”  process  can  make  data  more  useful  for  analytical  pur¬ 
poses.  No  editing  program  can  eliminate  the  small  random  error.  It  Is 
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cne  oig  nijtJK-  ,  j  iv  or  r  dty,re t  ini.’  ;•  t  inpt  ratun- ,  uhit_h 

needs  correction.  Since  the  influence  of  erroneous  records  on  results 
increases  as  the  length  of  the  observational  series  el  record  decreases, 
the  need  for  quality  assurance  is  the  greater  the  shorter  the  record. 

Tn  some  instances  censoring  of  frequency  distributions  by  eliminating 
extreme  values  may  save  eliberate  screening  procedures.  This  cannot  be 
applied,  however,  if  one  of  rha  .nn.nl  vs  is  fc  nls  is  th''  study  of  extremes. 

In  the  case  of  ndiosonir  data  a  second  re.-,aon  apainst  censoring  can  be 
pointed  out.  because  of  vertical  consistency,  data  elimination  at  oue 
level  without  attempt  of  correction  may  lead  to  discontinue  the  ascent 
from  the  censored  Level  up.  This  may  further  reduce  the  already  decreasing 
number  of  observations  with  altitude  and  nay  leave  very  few  data  reaching  a 
top  Level  of  km,  for  example.  Thus  the  cure  is  worse  than  the  disease. 

The  availability  of  high  speed  computers  has  opened  a  new  field  in 
applying  quality  control  method*  and  many  raethoda  considered  too  elaborate 
and  cumbersome  without  computer  use  can  now  be  employed  without  diffi¬ 
culties.  < 

Some  of  the  few  basic  principles,  which  reappear  and  can  be  counonly 
applied,  may  be  demonstrated  from  the  Army  Missile  Command's  screening 
program  of  radiosonde  data. 

Since  the  author's  detailed  article  is  already  scheduled  for  publi¬ 
cation,  (See  1969c)  only  some  basic  principles  will  be  presented  here. 
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_  _  .  _  .  _  .1 

1.1.  vut.u'.tra,  ne.ft.tuc  <..j 

Any  automated  screening  procedure  must  be  so  designed  that  particu¬ 
lar  (consistent)  errors  as  veil  as  inconsistent  errors  can  be  recognized. 
Ibis  goal  la  rendered  more  difficult  by  the  requirement  that  screening 
procedures  should  have  a  simple  logic  for  computerised  treatment. 

L.  Trivial  Errors  CherV 

Under  this  first  category  fall  all  errors  which  are  easily 
recognized,  and  in  naiy  Instances  an  automatic  correction  can  be  made. 

The  errors  can  be  divided  largely  into  three  groups:  coding  errors, 
data  and  limit  checks. 

In  the  first  group  one  may  encounter  errors  such  as  wrong  location 
number,  incorrect  elevation,  falsa  identification  coda,  mis takas  in  coding 
tha  type  of  observation,  erroneous  time,  etc. 

A  second  group  comprises  checks  for  completeness  (missing  data), 
duplication  and  sequence  of  the  records.  If  It  la  intended  to  supplement 
the  original  data  by  automatic  fill-in  procedures,  they  can  be  incor¬ 
porated  in  this  phase  of  the  screening  procedure  or  at  a  later  date. 

The  last  group  la  tha  limitation  violation,  e.g,  data  art  out¬ 
side  established  tolerance  limits  or  physical  boundaries.  For  example 
in  our  case  the  dew  point  temperature  cannot  ba  greater  than  the  air 
temperature  and  the  wind  direction  cannot  exceed  16  compass  points  or 
360  degreos. 

The  examples  given  for  the  Above  error  groups  ere  some  guidelines 
and  are  not  exhaustive.  They  serve  only  as  a  demonstration  for  tha  type 
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ClciiJ.y,  i  procedure  for  trivial  orrr,r  ri.r.-its  pvld#>nely 
depend a  largely  on  Che  type  o£  available  data.  The  correction  of  the 
deficiency  may  also  vary,  e.g.,  If  dealing  with  one  station}an  automatic 
correction  could  be  made  for  wrong  station  code.  In  other  instances 
elimination  of  the  date  nay  be  necessary.  Ii  one  had  to  establish  a 
map  by  computer,  this  may  be  the  only  way  tc  tvikcc  the  effect  cl  large 
errors,  while  for  other  analyses  time  and  personnel  may  be  available  to 
go  through  flagged  observations  and  to  painstakingly  check  their  validity. 
'u.  Error  Checking  by  Adjacent  Data 

In  this  group  incona 1st ancles  are  checked  against  adjacent 
data  or  a  fiald  of  date  in  the  horisontal  (map  or  equations),  vertical 
( cross-sections  or  equation*) ,  or  by  time  relationship.  The  checking 
procedure  depends  largely  on  established  physical  or  derived  empirical 
laws.  Again,  procaduraa  aim  at  flagging  suspicious  values  by  computer 
methods  or  correcting  them  if  such  procedures  can  bo  established. 

Tolerance  limits  of  differences  between  two  or  more  observations  must 
be  derived  first, 

a.  Horisontal  Checks 

This  type  of  ehacklng  process  can  be  applied  if  computerised 
maps  era  available  or  become  the  end  product  or  if  records  for  neighboring 
stations  for  the  same  period  of  record  are  given.  Under  physical  lava 
one  may  understand  conditions  like  the  gradient  wind  relationship  etc. 
Empirical  relatlonahlps  between  neighboring  stations  or  thresholds  of 
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i i f ;  l *.  .  *  d  ;  1 1 f* r*  '  jpt'ti  -s m f  •! ,ip-j  could  ~z  r^trhi' 

t',u3e  i'mpj.ricai  i-.n.uio. ishi.ps  are  derived  in  tabular  form  or  as  analyti¬ 
cal  expressions  is  not  important,  except  that  it  is  more  convenient  to 
work  with  mathematical  statements  for  which  computer  program! ng  is 
usually  very  simple,  changes  of  errors  era  considerably  lover  as  opposed 
to  table  inputs,  especially  uhpn  these  have  more  than  one  entry, 
b.  Vertical  Rela*  Unship 

The  l’,  s.  Army  Missile  Comanri  *3  procedure  of  screening 
r.id iosorniu  uit«  tuUes  n^uvilv  on  vertical  relationships.  Crosa-sectiono 
could  be  used,  but  only  if  they  are  readily  available  or  calculation  of 
the  cross-section  L.y  computer  methods  is  the  goal.  The  author  does  not 
know  of  any  program  at  tha  present  where  space  cross-sec dona  have  bsen 
utilized  lot  data  control.  Time-sac tions  have  been  employed  by  Canfield 
et  al.  (1P66). 

Our  program  checks  two  groups  of  elements,  thermodynamic 
quantities  and  wind.  In  the  thermodynamic  portion  the  lapse  rets  between 
two  consecutive  observations  at  different  altitudes  is  computed  end  com¬ 
pared  with  the  dry  adiabatic  lapaa  rate.  This  method  has  proven  quite 
efficient  and  satisfactory,  as  usually  any  error  in  pressure  or  tempera¬ 
ture  will  show  up  eventually  in  a  suparadiabatlc  lapse  rate  either  et 
ur  tested  date  pair  or  at  the  next  step.  For  example,  assume  e  10° 
negative  error  in  the  temperature.  If  it  is  the  higher  of  two  altitudes, 
it  creates  e  auperadlabatlc  lapse  rate.  If  the  error  is  positive,  one 
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would  have  an  Inversion  for  this  step  of  the  prop,!*  .nil  the  record  is 
not  flagged ,  The  next  step,  however,  would  give  ,1  aunerad  fabatic  lapse 
rate. 

The  last  observation  of  a  radiosonde  ascent  cannot  be 
checked  by  this  method,  as  there  is  no  other  observation  to  compute  the 
lapse  rate.  Thie  last  point  could  be  checked  oy  tolerance  limits  or  ot!i*i 
tools. 

It  should  be  added  that  superad tabetic  lapse  rates  are  not 
automatically  eliminated  by  our  program.  The  cause  can  be  mam  told. 

There  may  axiet  the  unusual  case  of  a  trua  auperadiabatic  lapse  rate  in 
nature.  One  may  have  a  temperature  or  pressure  error  nr  the  data  can  be 
out  of  sequence  by  erroneous  pressure.  Thus  all  "auspicious"  data  arc 
flagged  and  checked  by  a  qualified  meteorologist. 

Since  this  simple  tool  worked  to  well  for  the  thermodynamic 
parameter*  a  similar  principla  was  sought  for  the  wind.  In  the  beginning 
wind  data  were  checked  by  the  frequency  distribution  of  wind  shear  with 
technique#  .established  by  lasanwangar  et  el,  (1961).  This  is  usually 
ciuabereoat  end  expensive,  as  computations  of  frequency  distributions  are 
generally  costly.  The  difficulty  in  establishing  a  unique  relationship 
was  racognleed  by  ringer  et  el.  ( 1965)  vho  established  vertical  shear 
Unite  for  wind  checks  in  tabular  form  for  a  few  thresholds  of  layer 
thickness.  However,  their  method  requires  detailed  criteria  depending 
on  layer  thickness,  wind  epeed,  end  difference  of  direction  or  speed  of 
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'■  t:  i  «/'  .  :  .  <"!  «>  *"fhc-r  *  if*-.-;;  ha;  d.ui.eu  a  r  ei  auxonsnip 

between  'he-  vector  che.-,r  (Av)  and  the  shear  interval  (Ah) 

B  • 

Av  a0  (Ah)  1  (  1) 

The  exponent  for  extreme  value  was  found  to  be  1/3  ( see  also  Ensenvanger 
and  Reiter  L<-^9a)  ,  For  n.;e  in  our  program  eqn.  (  l)  had  to  be  modified 
to  aceoawod.ito  r,o  mal  ■  >ed  shear  interval,  thus  Av  “  V  Ah,  resulting  In 

Vp  ™  e  (Ah)  “J  (2) 

Where  V  denote?  the  total  vector  shear.  With  a  ■  2.3,  e  reasonable 
threshold  V  "(  m  sec-1  per  Interval)  is  found.  All  values  exceeding 
V  are  flagged. 

Equation  (a)  expresses  e  unique  relationship  similar  to  the 
lapse  rate  sa  n  convenisnt  and  simpla  toleranca  critaria, 
c.  Tima  Serias 

All  alaaanta  showing  some  form  of  tima  ralatlonahlp  could 
be  checked  by  methods  taking  advantage  of  this  relationship.  It  does 
not  matter  whether  the  time  relationship  is  periodic  or  aperiodic. 
Howevar,  in  all  time  related  checking  procedures  the  time  relationship 
must  be  establishad  first. 

In  case  of  periodic  variations  it  is  quite  convenient  to 
represent  records  by  a  Fourier  series  and' check  an  expected  versus  an 
observed  value.  A  tolerance  limit  for  a  maximum  (absolute)  difference 
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from  the  expected  value  may  be  determined  by  statistical  methods  of 
error  theories.  Sometimes  it  may  be  quite  sufficient  and  suitable  to 
use  subjective  tolerance  limits. 

If  an  aperiodic  time  relationship  (e.g.  persistence)  has 
been  found,  tolerance  criteria  for  time  differences  can  be  employed. 

In  all  cases  an  expected  value  is  tested  against  the  observation. 

A  time  checking  procedure  can  be  applied,  even  If  no 
functional  relationship  can  be  found.  Although  time  differences  may 
be  randomly  distributed,  a  tolerance  criterion  can  be  developed  similar 
to  that  described  in  e  later  chapter  on- frequency  distributions,  if 
the  difference  exceeds  e  csrtaln  magnitude,  it  may  indicate  en  erroneous 
observation. 
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-  * 

Frequency  ..  tr:  t  ut  ion  Chocks  ' 

Altnoug.i  met.iuj;,  described  in  the  previous  auctions  should  catch 
the  bulk  of  errors,  some  mistakes  may  slip  through.  Let  ua  assume  that 
the  surface  obaervat ier  of  a  radiosonde  ascent  is  missing.  Vertical 
consistency  could  not  discover  this  mistake.  Although  it  could  have 
been  flagged  in  the  trivial  error  el’tc-c.  --ther  examples  can  be  given 
where  vertical  cons  stun  •.  existed,  but  t'.,  total  ascent  was  either  too 
warn  or  told.  Ties*  er-/rs  can  be  checked  against  a  frequency  distribution. 

In  the  Army  Missile  Command 'a  earlier  screening  procedure  pre¬ 
liminary  trequency  distributions  were  established,  with  printout  of  the 
first  five  maxima  and  minima,  mean  and  standard  deviation.  Visual 
inspection  of  the  frequency  distribution  then  revealed  isolated  obser¬ 
vations,  Vertical  profiles  for  the  maxima  and  minima  vel.re  drawn  and 
auspicious  records  could  be  detected  by  irregularities  in  profiles. 

This  process  was  time  consuming,  and  not  coo  many  erroneous 

ascents  were  discovered,  since  the  majority  of  corrections  had  been 

made.  Nevertheless,  all  frequency  distributions  had  to  be  inspected. 

This  phase  of  the  program  was  costly,  too,  as  frequency  distributions 

had  to  be  grouped  by  small  class  intervals  to  detect  isolated  records 

and  class  intervals  shifted  from  month  to  month  or  by  altitude.  This 

phase  of  the  program  was  modernized  by  utilizing  only  mean  and  standard 

deviation  and  selecting  suspicious  values  by  a  predetermined  threshold 

x  ,  to  be  exceeded  only  a  certain  percentage  of  the  time.  This  eliminates 
tn 

the  establishment  of  frequency  distributions  and  reduces  the  printout  aa 
only  flagged  observations  appear. 
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li  is  evident  ehac  correct  as  well  <ib  Incoricct  observations 
will  be  i lagged  and  printed  out,  as  one  should  expect  a  number  of  obser- 
vations  exceeding  the  threshold  x^  In  agreement  with  the  selected  per* 
centage  figure.  Unfortunately  there  is  no  easy  way.  to  separate  the  two 
groups  by  computer  methods,  es  large  deviation  can  be  caused  by  extreme 

weatner  events.  All  cases  muse  bt  lodged  by  the*r  own  merits.  It  is 

/ 

reiterated  that  acceptance,  correction  or  deletion  of  >in  observational 
record  depends  largely  on  the  purpose  of  any  analysis  and  existing 
posaibilitiea.  We  have  found  it  quite  convenient  to  make  available  for 
eny  flagged  value  the  threshold  for  99%  end  the  frequency  of  occurrence  „ 
which  the  flagged  observation  would  have  in  a  theoretical  distribution 
law  .  These  valuer  ere  helpful  guidelines  for  evaluation,  but  are  generally 
not  sufficient  by  :hem*eive.s  for  a  'decision  that  the  observation  Is 

^  i 

t  1. 

erroneous  or  not.  It  should  further  be  pointed  out  that  censoring  of 
the  frequency  distribution  cannot  be  applied  in  our  particular  case. 
Especially  extreme  value  date  analysis  is  pert  of  the  subsequent  research 
topics.  Censoring  would  not  solve  the  quality  assurance  problem, 
a.  Gaussian  Distribution 

Ihe  critical  problem  is  the  determination  of  the  threshold 
x^.  outside  of  whose  boundary  observations. should  be  flagged.  In  statis¬ 
tical  terms,  one  has  to  select  a  certain  point  of  the  cumulative  distri¬ 
bution  on  one  or  both  sides  of  this  curve.  The  computation  of  the 
cumulative  distribution  la  cumbersome  for  most  types  of  distribution  laws 


133 


*s  u  evolves  vite.;,.  ii.-ig  frequency  density  functions.  In  the  Army 
Missile  Command' p  earlier  version  empirical  cumulative  distributions 
were  computed  to  secure  close  agreement  with  the  observed  frequency. 

This  had  the  advantage  of  the  frequency  curve  being  independent  from 
the  statistical  type,  or  the  mean  and  the  standard  deviation  of  the 
distribution.  Later  this  wpr  replaced  by  establishment  of  frequency 
distributions,  which  display  less  complexity  in  computer  programing. 

If  t’ie  t.er  it  follows  cn  approximate  Gaussian  normal 
distribution,  one  could  determine  the  threshold  by 

*th  *  *  t  (3) 

where  x  Is  the  mean  value,  o  the  stendard  deviation  and  the  coefficient 
'"a"  would  be  determined  by  the  desired  percentage  exceedence,  e.g.  a  -  3.0 
for  .135$  of  the  observations  beyond  that  point.  All  observations  above 
xth  woulc*  then  be  flagged  and  printed  out. 

Since  the  relationship  between  the  cumulative  distribution 
and  the  standard  deviation  is  known  for  ths  Gaussian  distribution,  the 
establishment  of  thresholds  should  not  crsste  any  problem  for  meteoro¬ 
logical  elements  following  this  distribution  law.  Gaussian  laws  apply 
to  most  thermodynamic  quantities. 

b.  The  tfeibuli  Distribution 

If  Eqn.  (3)  were  applied  to  meteorological  elements  not  in 
agreement  with  the  Gauasian  law,  one  would  have  either  too  many  flagged 
observations  or  not  enough,  depending  on  the  deviation.  Since  the 
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relationship  between  standard  deviation  and  cumulative  distribution  for 
other  types  of  distributions  is  complex  and  generally  cannot  be  found  in 
simple  tables,  the  ideal  solution  would  be  a  cumulative  frequency  lav 
versatile  enough  to  adjust  to  a  variety  of  types  with  good  approximation, 
thus  we  applied  the  Welbull  distribution  with  considerable  success  in 
our  screening  procedure. 

Hie  Welbull  distribution  is  defined  as  a  cumulative  type 


P(  x)  a  1  •  e 


(M 


with  Q  and  fl  do  the  reference,  scale  and  shape  parameter,  respectively. 

Any  percentage  F(  x }  can  be  related  to  by  the  modification  ot  equation  (U) 
•  to 


e 

Xth  "  Q\fn  P  +  7  (5) 

where  P  «  1  /  ( 1  -  F(  x) )  (5a) 

The  estimation  of  the  parameters  is  the  only  difficulty 
left.  Maximum 'likelihood  estimation  for  all  three  parameter!  cannot  be 
performed  analytically  and  solution  ia  very  time  consuming.  Thus  the 
utilisation  of  the  maxlana  likelihood  method  for  three  parameters  would 
have  Increased  costs  compared  to  frequency  distributions.  Simpler 
methods  exist  whan  7*0  ( aea  Kao,  1950  or  Menon,  1963),  however,  the 
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.if  sit.  ?  - 1>  r.  •>  -  vouid  rocucc  the  flexibility  of  adjustment  for  the 
Weibull  distribution  and  would  limit  the  ability  to  fit  the  frequency 
distribution.  $d,cee  the  major  goal  in  the  checking  procedure  is  the 
establishment  of  a  threshold  value  the  reader  may  find  a  parameter 

estimation  by  moments,  developed  by  the  author  (1968,  1969b)  quite 
convenient. 


y  ~  -  jab  ♦  a",)/(b  -  a*2)^"  ( fcaj 

v 

y  denotes  the  skewnese,  the  ratio  of  the  third  moment  (reference  mean) 

l 

to  the  cube  of  tne  standard  deviation,  y  *  u  /d3 

1  s 

Since  a,  b  and  c  depend  on  p  only,  a  computer  solution  of  (6«)  ie  rela¬ 
tively  i  asy  or  tables  can  be  used  ( aee  Easenwanger,  1968,  1969b) 


a  -  r  (1  +  1/e)  (7a) 

b  -  r  ( i  +  s/e)  (7b) 

c  ■  r  (l  .♦  5/B)  (7c) 

With  0  known,  the  other  parameter*  become 

9 £  -  o2/(b  -  a2)  (6b) 

and  7  ■  x  -  0‘a  (6c) 


The  three  momenta  of  the  distribution  muet  be  known  for  application  of 
eqns .  5,  6,  and  7.  In  two  ca*ee  two  moments  ere  sufficient. 
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As  shown  by  the  author  ( 1968)  the  y  can  he  approximated  by 

1 

7  -  L.404TE  -  .0646c  +  .0987  (8a) 

i 

for  wind  and  by 

y  -  3.12231  -  .3^80a  -  .U31*S  (fib! 

for  the  total  vector  wind  shear 

E  *  c  /c3  *  x  ( 1  ♦  3d  +  Id2)  /o'3  '  Oa) 

3 

with  d  +  1  ■  o2/x  (9b) 

The  aeeond  case  employa  the  Welbull  distribution  for  elements  whose  dis¬ 
tributions  follow  the  Gaussian  law. 

Thus  p  pan  be  determined  a  priori.  Eqn.  (6a)  gives  ?  **  5.60. 
If  the  squared  difference  of  the  Geussien  end  the  Welbull  distribution 
at  steps  of  half  a  standard  deviation  0  within  +  3.5®  is  computed  and 
sunned  up,  a  minimum  is  discovered  at  0  ■  3.55.  Table  la  exhibits  the 
frequencies  for  the  Gaussian  and  the  Welbull  distribution  (cumulative  at 
left  and  density  at  right)  for  0  ■  3.55.  All  differences  are  Less  chan  If. 
The  last  columns  in  both  sections  contain  the  differences  for  the  0,  if 
selection  is  made  for  the  smallest  possible  maximum  deviation  of  any  fre¬ 
quency  within  +3.5®  range. 

Of  more  importance  may  be  the  agreement  between  the  x-valuea 
as  these  are  used  to  establish  the  flagging  limit  of  Eqn.  (3).  The 
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Table  l.  Comparison  of  Weibull  Distribution  With  Cjossfin  Distribution 
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Table  1.  (Continued. 
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Weibull  distribution  for  half  units  of  a  within  +  W  lies  at  p  ■  h.36 
(Tabic  lb).  It  should  be  noticed,  however,  that  the  differences  for  the 
two-sided  fit  increase  towards  the  marginal  classes.  This  is  a  handicap, 
but  Is  acceptable  since  it  is  preferable  to  flag  more  than  the  expected 
number  of  observations  rather  then  less.  The  threshold  could  be  adjusted, 
too.  Again,  if  a  minimum  of  the  absolute  deviation  is  desired,  one  would 
select  p  »  4.26  with  deviations  smaller  than  0.60  at  tm-:  ends. 

Since  It  is  known  whether  an  observation  is  below  or  nbove 
the  mean  value,  e  one  aided  fit  solves  ths  problem  of  poor  agreement 
towards  the  ends.  Good  approximation  for  the  minimum  threshold  can  be 
obtained  with  e  3  of  9.33  or  3.^-,  while  one  may  aalect  a  3  of  2.96  or 
2.97  for  the  maximum  end.  The  differencea  are  displayed  in  the  right 
portion  of  Table  lb. 

The  advantage  in  using  the  Weibull  distribution  for  flagging 
Instead  of  the  concept  of  the  normal  distribution  lies  in  the  easy  compu¬ 
tation  of  related  frequency  value*  for  the  flagged  observation  with  Eqn.  ( 5) 
and  (?a).  This  eliminates  any  tabular  input  as  necessary  for  Bqn.  (3)  end 
one  program  can  be  applied  to  ail  types  of  frequency  distributions, 
c.  Elements  With  Various  Types  of  Distributions 

Thermodynamic  quantities  and  wind  can  be  treated  with 
techniques  as  outlined  previously.  The  Weibull  distribution  is  very 
flexible  and  thus  can  be  utilised  for  ths  purpose  of  flagging  for  numerous 
elements.  Soma  distributions  may  display  untolerable  discrepancies. 


Transformation  of  seals  sometimes  helps,  such  as  a  logarithmic  progression 
of  visibility  data.  This  must  be  left  to  the  individual  analyst.  Hie 
Welbull  distribution  is  very  flexible  and  transformation  can  usually  be 
avoided . 


•or.t-  ■  f  Caution 


It  is  reiterated  that  no  quality  assurance  program  can  make 
good  data  out  or  bad  r -'curds.  These  programs  con  only  contribute  to  an 
’’editing"  of  date,  u'ter  which  the  larger  errors  ( hopefully)  have  been 
eliminated.  Since  these  large  errors  can  bias  any  statistical  or  computer 
result,  the  correct! m  of  rhese  obvious  mistakes  is  naceaaary.  It  must 
b.1  cautioned,  linv^m  i’hl  correct  ion  methadc  cannot  hr  geared  to  an 
HxpetLcd  .via’.ys  H  result,  us  all  observations  contradictory  to  an  assumed 
hypothesis  to  be  teBted  by  these  data  are  then  eliminated.  Correction 
methods  must  be  independent  of  subsequent  analysis.  One  cannot  check 
persistence,  for  example,  if  the  majority  of  data  have  been  filled  in 
by  methods  derived  from  persistence. 

The  editing  process  by  "experts"  is  usually  cumbersome,  but 
correction  methods  by  computers  must  be  carefully  designed.  Where  con¬ 
sistency  equations  can  be  obtained,  methods  for  random  error  corrections 
can  be  developed  with  the  complexity  of  the  atmosphere  it  la  difficult, 
however,  to  pinpoint  unequivocally  differences  between  e  rare  event  end 
an  obvious  mistake. 

Any  correction  method  should  be  based  upon  known  or  derived 
principles  of  error  sources.  Sometimes  data  are  rectified  which  later 
prove,  correct  In  the  light  of  expanded  knowledge. 

Establishment  of  threshold  values  is  arbitrary.  Threshold 
values  must  be  designed  to  catch  all  the  large  arrora  without  the  burden 
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of  reviewing  too  many  data  by  tin-  expert.  Any  '.itnt  a  large  pale  of 
flagged  data  appears  for  a  particular  data  sample,  a  search  for  a 
systematic  error  should  precede  any  detailed  correction  operation. 

This  systematic  error  can  then  be.  corrected  befor  other  computer 
runs  are  made,  Sometimes  a  big  bulk  of  printout  can  be  caused  by 
improper  selection  of  the  'threshold.* .  Thon  i n  adjustment  will  ^ive 
reasonable  .amounts. 

It  should  be  further  mentioned  that  selection  of  thresholds 
succeeds  for  unlimited  distributions  only.  It  would  be  absurd,  ,for 
instance,  to  flag  all  calms  in  surf. ice  wind  distributions  or  all  dry  • 
records  for  precipitation  data.  Elements  with  u-shaped  distributions 
could  in  general  not  be  checked  by  frequency  methods. 

K 
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ltt.  rv.NCL'.S'u,:.. 

Analytical  methods  Cor  the  editing  of  observational  data  have 
been  divided  inn  tl> rt e  aa ior  groups,  the  checking  of  inconsistencies 
( trivial  errors^ .  the  procedures  employing  a  set  of  data  with  inter- 
relationship,  and  utilization  of  frequency  distributions,  the  methods 

presented  may  serve  as  r  guideline  and  cannot  be  exhaustive,  as  the 

/ 

complexity  c>£  tb<*  .it  ctpl  'm  with  Tts  different  me tenr.i logical  parameters 
necesritatef  indvidu  1  techniques  depending  on  the  treated  element, 
the  three  described  groups  of  error  checks  are  conraon  with  any  quality 
assurance  program. 

It  is  repeated  that  editing  of  data  cannot  replace  a  carefully 
carried  out  observational  program  with  adequate  instrumentation.  One 
can  assure,  however,  that  large  mistakes  and  systematic  errors  from 
various  sources  are  discovered  and  any  bias  of  <he  results  due  to 
erroneous  data  is  largely  reduced.  The  small  random  error  cannot  ordi¬ 
narily  be  eliminated. 

Although  the  data  may  have  gone  through  quality  assurance  programs 

3  ,  1 

several  times  before  they  reach  the  investigator,  it  is  neverthelsss 
advisable  to  resubmit  the  data  to  a  screening  procedure.  Editing  of 
data  by  other  Investigators  or  installations  does  not  automatically 
guarantee  that  the  received  data  are  free  of  mistakes. 

The  goal  of  the  editing  process  should  not  be  to  correct  nature 
and  reject  data  which  do  not  fit  into  a  predetermined  model  or  hypothesis, 
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i  i*.iier  k  .•» . .  t.iif  'Jat.j  should  „•  t-i.se.  upon  known  «:;■  d - 

covered  source.  of  crvor  only.  If  the  latter  L-.  ke?c  in  mind,  analyti¬ 
cal  methods  of  quality  assurance  will  serve  their  useful  purpose. 
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ABSTRACT 


A  STATISTICAL  MODEL  FOR  THE  ANALYSIS  OF  SIMULTANEOUS  TWO- 
STATION  IONOSPHERIC  SOUNDINGS 


Dr.  Erwin  Blser  Mr.  Richard  D'Accanii 

US  Army  Electronics  Command,  Fort  Monmouth,  NJ 

1.  Ionospheric  sounder  data  characteristics  change  as  the  distance  between 
two  sounder  stations  ie  increased  from  0--500  Km.  It  is  therefore  desirable 
to  know  about  the  degree  of  correlation  one  can  expect  between  vertical  in¬ 
cidence  (single  station)  data  and  oblique  incidence  (two-station)  data.  It  will 
be  shown  that  a  single  ionospheric  sounder  {innosonde)  operating  in  the  ver¬ 
tical  incidence  mode  can  provide  useful  data  over  an  area  of  60  Km  radius. 

2.  Experimentation  was  performed  in  the  2-16  MHz  frequency  range  using 
two  ionosondes,  one  as  a  fixed  terminal  and  die  other  as  a  mobile  terminal. 
Each  terminal  made  scheduled  soundings  every  ten  minutes  from  0530  to 
17.10  hours  for  ten  days.  While  the  fixed  terminal  was  transmitting  and  re¬ 
ceiving  its  own  signal,  the  mobile  terminal  would  simultaneously  receive 
the  same  transmission;  likewise  for  the  mobile  with  respect  to  the  fixed 
terminal.  As  each  ionosonde  transmitted  and  received  in  the  vertical  in¬ 
cidence  mode,  the  other  sounder,  receiving  the  same  transmission,  com¬ 
pleted  the  obilque  ionospheric  mode.  (An  oblique  mode  or  path  is  one  be¬ 
tween  two  stations  space  a  distance  apart;  a  vertical  mode  or  path  occurB 
when  either  station  receives  its'  own  transmission. ) 

3.  The  experiment  was  designed  primarily  for  a  paired  difference  model, 
that  is,  the  pairing  of  data  occurred  as  planned  by  the  experiment.  The 
data  were  also  analyzed  by  a  paired  comparison  method  to  focus  on  the  gain 
of  information  achieved  with  the  paired  difference  or  randomized  block  design, 
and  to  show  that  vertical  incidence  and  oblique  incidence  ionosonde  data  are 
good  estimators  of  each  other  over  short  distances. 

4.  The  application  of  a  similar  method  of  analysis  will  hopefully  be  used  in 
future  experimmtation  to  substantiate  a  high  degree  of  correlation  between 
vertical  and  oblique  incidence  soundings  over  field  army  distances  (0-300  Km). 


This  article  has  been  reproduced  photographically  from  the  authors  manuscript. 
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The  objective  of  the  present  data  analysis  is  to  shjv  that  daily  ionospheric 
soundings  taken  at  vertical  incidence  (VI)  are  very  nearly  the  same  as 
oblique  incidence  (OI)  soundings  taken  over  a  60  Km  path  (see  Figure  1). 
We  are  interested  in  formulating  hypothesis  tests  to  determine  whether  or 
not  the  Vertical  Incidence  data  (population  I)  is  nearly  the  same  or  is,  in 
fact,  identical  to  thn  Oblique  Incidence  data  (population  Q).  The  analysis 
investigates  a  total  i  f  85  daily  measurements  of  critical  frequencies  per¬ 
formed  over  a  nine-day  period,  taken  every  ten  minutes  from  0530  hourB 
to  1930  hours,  for  a  60  Km  path  (see  Figures  6,  7).  This  yielded  nine 
observations  of  critical  frequency  per  time  slot.  Samples  of  raw  data 
appear  in  Figures  2,  3,  4,  5.  In  order  to  test  whether  or  not  a  given  hypo- 
thesis  ie  supported  by  a  set  of  data,  we  devised  a  rule  of  procedure  depen¬ 
dent  upon  certain  calculations  obtained  from  a  sample  of  the  data,  and  de- 

(3) 

cided  to  accept  or  to  reject  the  hypothesis  formulated  ,  Two  experiments, 
E}  and  Eg  were  used  in  comparing  the  means  of  population  I  (VI)  and  those 
of  population  n  (OI)  .  The  homogeneity  of  variance  was  tested  by  the  use 
of  the  F  test,  where  o02  was  compared  to  cv2  . 

To  test  homogeneity  of  variance,  the  variances  of  the  vertical  and  oblique 
incidence  data  were  paired.  The  85  grouped  values  were: 
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Tests  of  hypotheses  for  the  equality  of  two  variances  were  formulated  id 
follows: 

(2)  Ho:ffo2  s  °v2  ve-  lh-°o2  *  *v2 

or:  2  2 

1I_:  ^  =1  vs. 

0  ffv  °vi 

o 

Qq- 

The  rejection  region  is:  0y2  *  k,  where  k  is  found  by  specifying  the 
significance  level  a  *  .  01  .  The  following  probability  function  describes 

the  relationship: 


ft  2  /'!' 

Under  the  null  hypothesis  HQ,  (  has  an  F  distribution  with  (n-1), 
(n-l)  degrees  of  freedom,  whic h  results  in  k»F|jn-l  ),  (n-l);  (1-|)J  and 
the  rejection  regions  are: 

g  2 

•g*7  a  F[(n-1),  (n-l);  (1-|)3 
(4)  V 

-  2 

HOy  a  F[(n-1,  (n-l);  2]  for  a  »  .01 
av  2 

n  9 

If  these  inequalities  are  satisfied  by  SQ‘  and  Sy  ,  then  we  can  conclude 
that  the  estimated  variances  are  significantly  different  at  a  *  .01  level  of 
significance.  That  is  to  say,  H0  is  rejected  when: 
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1 


(5) 


2 

r  iit.-i),  (  n-1 ),  %  ]  a  ^2  £  FC(n-l),  (n-l);(l-*)] 
6  v  2 


By  letting  S0^  nnc!  S  '  ,  the  sample  variances,  estimate  a 02  and  crv2, 

(1 > 


we  form  the  F  ratio; 


(6) 


F  * 


§4 

2a: 


17 

'V 


which  has  the  F  distribution  with  (n-l),  (n-1)  degrees  of freedon^. 

“\  o  1 

F  involves  the  ratio  but  is  independent  of  c02  and  <rv2,  therefore: 

<V 


(7) 


PrLF[(n-l),  (n-1);  §]  * 


<  F[(n-1),  (n-l);(i-f)lJ«i-« 


In  testing  the  means  the  observations  were  grouped  into  85  values  for 
each  experiment: 

(8)  Ej  ■  fdj,'  d2,  ,  dgsJ 

9 

where  dj  -  xQ  -  yv,  and  d  “(l/n)!^,  i  ■  1,  .  .  .  .  ,  9  per  time  slot. 


(9) 


e2  s  ffll»  fi2'  *  1  •  '  wher®:  “  *0j  '  yv.  Per  time  slot 

J 
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These  sets  consist  of  the  mean  values  of  the  differences  between  OI  and 

•j 

VI,  (Ej),  and  the  differences  between  the  means  of  the  two  populations, 

(E2),  repeatedly  taken  at  the  same  time  daily  for  nine  days.  Theae  two 

2  2 

sets  are  assumed  to  be  normally  distributed  with  variance  crd  and  Op 
so  that  the  means  of  differences,  dj,  and  the  difference  between  the  means, 
are  normally  distributed.  Since  cd2  and  cp2  are  unknown,  take 
estimate!  of  the  variances  for  each  time  slot  for  Ej  and  E2  are: 

2  i  9  _  2 

(10)  Sd<  ■  r**r  T  (d^  -  d)  for  E,,  where 

1  n- i  *  * 


an 


n  *  9  samples  per  time  slot, 

d  ■  mean  of  the  differences  between  OI  and  VI  per  time  slot. 

\ 

dj"  difference  between  and  fVI  ■  x0i  -  yv  . 


*or\.  s  2  ,  hi-II^Mnj-DS,8 

P  n1+ n2-2 

where  n^  ■  n2,  and: 


9 

*  <*n. 

1  i 
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Sp  *  pooled  estimate  of  variance  for  E2 
n  *  9  samples  per  time  slot 

x0,  yv  ■  means  of  OI  and  VI  populations  per  time  slot. 
x0,  yVi  r  OI  and  VI  data  per  time  slot. 

(2 ) 

The  t-statistics  ernplc,  e«J  are: 


whore  n»9  samples  per  time  slot,  and  ni»n^n2*2B16 

degrees  of  freedom 

Sp  ■  «/Sp2  ,  the  pooled  standard  deviation  for  E2, 

Sdj*  -s/S^2  ,  the  standard  deviation  for  Ej  . 

Therefore,  the  populations  are  t-distributed  with  (n-1),  and  n*  degrees 
of  freedom.  The  first  experime  nt  or  "paired"  difference  test,  E^, 
concerned  itself  with  analyzing  the  means  of  the  differences  between  OI  and 
VI  data,  The  second  experiment  or  "paired"  comp  arisen  test.  Eg  was 
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.■nru'rrnpri  with  in:i lv? i n?  the  dif fertmce  between  t.hi»  means  of  ihp  two 
populations.  For  E1  the  following  hypothesis  wus  formulated; 

(13a)  H,V  Mo  =  Mv  V!i •  lll;Mr, 

G 

where  d,  »(l/n)I  Uu.  -  yv.)  lor  each  time  slot, 

1  j  j  i  i  Vi 

For  En  the  following  1\.  inthesi  -  wnj  formulated; 

(13b)  IT0:  Mo  =  Mv  vs.  Ht:  M0  t  Mv 

where  fl  =  xQ  ■  y  v  for  each  time  slot, 

9 

and  x0  *(l/n)  ^  x0  , 

S3 

yv  --<i/n>js1  yVi  , 

and  xQ  and  yv  are  oblique  and  vertical  incidence  data  respectively, 
That  ie  to  say,  we  will  tost  a  null  hypothesis  H0,  (that  dj  or  flj  *  U) 

vs.  H-  .  If  we  accept  the  hypothesis,  this  would,  of  course,  indicate 

/ 

that  the  difference  between  01  -  VI •  0  for  each  time  slot  at  a  *  .  01  , 

If  we  assume  the  alternate  hypothesis  to  be  true,  then  the  01  and 

VI  data  would  be  significantly  different. 

(2) 

The  critical  region  of  these  tests  are: 

(14)  - U- - >  t  C(n-l);  a/2]  for  Ej,  which  can 

be  written  as: 


132 


ri  fry 

tC(n-l);  Cf/2]  >  - >  t[(n-l);  (l-a/2)] 

5“i 

tailed  test,  and 


for  a  two- 


3c  “  V 

"  ft  „  ;  v-  >  t[(n*);  a/  2]  for  E«  which  can  be  written  aa: 
■?,  Sp  <JTfn  2 


tC<n* );  tt/2]  >  *°  ■  >  tC(n');  (l-o>/2)]  , 

Sp^T/n 


where  n*9,  n*a(2n-2)»16  degrees  of  freedom,  and  with  n*9,  (n-l)a8 
degrees  of  freedom.  This  Indicates  that  if  (14)  is  satisfied  by  dj  and 
Sd^  and  (15)  is  satisfied  by  6}  and  Sp,  the  tests  are  rejected  under 
the  null  hypothesis  H0  and  that  d*  and  fi*  does  differ  significantly  from 
"0"  in  the  critical  region  (region  of  rejection). 

The  critical  regions  can  be  explained  by  the  following  probabilities:^) 

(18)  Pr[  1  >  tC(n-l);  a/2]]  ■  a  for  Ej  which  can  be 


written: 


Pr[t[(n-1);  a/2]  >  - >t[(n-l);  (l-a/2)]]  -  a. 


(17)  Pr[  '  ~  tf  n*:«/  2]1  ■  a  for  E*  which  can  be  written: 

*■  Spv/2/n  J  4 
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Pr;  tLii*;  Cut  21  >  t[n*,  (1  -Of/ 2 >  j  j  =  r;  , 

"  bpv/2/n 


where  a2.  01‘  is  the  pre-determined  critical  region,  ci  region  of  re- 

v. 

.lection.  Regarding  the  comparison  of  means  and  of  variances,  if  the 

1  *i 

null  hypothesis  H0  is  found  to  fils?,  then  the  power  function,  r, 
would  be  used  to  find  the  probability  that  the  alternate  hypothesis  Hj 
will  fall  completely  in  the  critical  region.  Let  ft  =  region  of  acceptance 
of  the  alternate  hypothesis  Hj  ,  Normally  TT  =  1-/S  should  be  very  large 
or  ft  vary  small.  To  illustrate  the  concept  of  a  rejection  region,  suppose 
we  have  the  following  hypothetical  probability  density  function  of  a  variable 


•>r{X/H0) 


a  region  of  rejection  for  H 


region  of  acceptance 
for  HD 


Illustration  (a).  PrfX/H^l  vs.  X 


(1»)  \  PrfX/H0]  dx»a 

A  • 

Therefore,  if  H0  is  true  (so  that  X  has  the  probability  distribution 

Pr{X/HQ3),  the  probability  of  a  random  observation  falling  in  the  critical 

(4) 

or  rejection  region,  X  >  Xc  is  Of,  that  is:  Xc  satisfies  illustration  (a). 
Now  consider  Hj  true  and  X  having  the  density  function  Pr{X/Hj}  .  The 
probability  of  a  random  observation  falling  in  the  acceptance  region 
(illustration  (b)  ),  X  <  Xc  is  (l,  that  is: 


-  Xc 

/  «  ,  ,  »  *  r^r^rirtTi  a 

\L'i*  ^  i  t  la/  uj  ;  Uy  -  p 


In  addition  to  hypothesis  testing,  the  analysis  estimates  intervals  I  and 


I*  for  which  we  can  expect,  with  99%  confidence,  that  €  I,  and 


S  ^ 

%  €1' 
Sy2 


That  is,  we  utilize  the  information  at  each  time  slot  using  the  t  tests 
described  in  equations  12,  and  place  a  99%  confidence  bound  on  the  true 
state  of  nature  at  these  points,  i.e,  d.  =  (x0i  -  yv^)  for  E1  and 

*  (x0j,  -  7V,)  for  E2  respectively.-  This  means  that  if  experiments 
and  E2  were  to  be  performed  say,  100  times,  we  could"  be  confident  that 

i  "  •  ' 

/ 

99%  of  such  intervals  will  contain  the  true  state  of  nature  at  each  time  slot. 
Thus  by  putting  confidence  bounds  bn  each  set  of  data  points,  we  would  have 
85  upper  and  lower  bounds  which  would  generate  an  envelope,  From  this 
envelope  we  can  conclude  that  for  the  spectrum  of  data  generated  in  this 
experiment,  we  are  99%  confident  that  the  data  will  be  contained  With  the 
envelope. 
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Interval  estimation  aids  in  obtaining  limits  C.  whirh  a»'<»  functions 

“  i  * 

of  the  sample  values  {fc)  or  functions  of  the  sample  values  and  known 

Sn* 

f  °  •> 

population  parameters  {d^},  {fij}  and  1^2/  .  The  limits  are  determined 

(5) 

so  that  the  probability: 

<21 )  Pr(ci  <  8  <  e2)  »  1  *a 

where  0  is  the  parameter  being  estimated  and  (1  -a)  is  the  confidence 
probability.  Consider  Die  problem  graphically,  where  f^(0)  and  f2<0) 

A 

are  drawn  so  that  Cj  €  ^(0),  c2  €  f2<6)«  Pr[fi<0)  <  0  <  f2(0)]  *1  and  0 
is  a  sufficient  estimator  of  0  obtained  from  the  data. 


The  line  segment  will  intersect  0  ■  0O  (true  value  of  parameter) 

if  and  only  if  fa*0o*fb.  TWs  Is  to  8®y  t*18*  Pr(fa«&*fb)  =  1  -  o;  this  is  also 
the  probability  that  (ci,c2)  includes  0O  . 


It  will  be  shown  subsequently  that  we  can  be  99%  confident,  (1-a  *.  99), 
that  F*  and  t*  will  be  between  the  calculated  upper  and  lower  limits 

i  / 

of  the  confidence  interval,  In  the  paired  difference  test,  the  probability  of 
accepting  li0: 

(22a)  Pr[t[(n-l);o/2]<  t*]  <  t[(n-l);  (1  -0f/2)l j  ■  1-a 

^  d  • 

where  t*^  «  — - .  From  this  equation  and  that  of  (16)  we 

sdj[ 

obtain  the  confidence  interval: 

(23a)  [di  ±  t[(n-l);a/2] 

This  means  that  we  can  be  100  (1-a)  %  confident  that  this  interval  contains 
d  *  0  under  H0.  (The  critical  region  ia:  t*[(n-l);a/2]  >  t*  >  t*  [(n-l);(l-a/2)]). 
Likewise,  for  the  paired  comparison  test,  the  probability  of  accepting  H0  1b: 

(22b)  Pr^tCn’;  a/2]  <  t*2  < ,t[n'j  (l-a/2)] j  •  1 -a 

•  i  t  1 

where  t*2  =  — 

s  pJTR 

(2) 

This  equation  and  equation  (17)  lead  to  the  confidence  interval:' 
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(2.ilJ 


i  a/2]  Sp/STnj 


The  F  test  for  the  variances  as  given  in  equation  (7)  can  now  be  rewritten: 


(24)  Pr[(^%)  F[(n-l),  (n-1);  a/2]  < \<  (|^  F[(n-1),  (n-1  );(!-<*/ 2)]] 


«v 

1  -  a 


where  the  confidence  interval  is: 


[(^i)F[(n-l).  (n-iy2],(^)F[(n-l),  (n-1).  (l-a/2)]] 


The  probability  that  will  be  contained  within  this  interval,  under  Hc 

Sy2 

is  (1-a). 


For  Ejj,  the  paired  comparison  teat,  the  computed  value  of  t  used  to 
test  the  hypothesis  at  10:00  is  0.  0174,  (see  Figure  (16)).  The 

corresponding  confidence  interval  for  the  same  time  slot  is:  I"{-0.  9405, 0.  9293). 
Note  that  the  interval  is  quite  wide  considering  the  small  difference  be¬ 
tween  the  sample  means  (for  10:00  hours).  Examination  of  the  data,  Figures 
8,  9, 10.  li,  show  a  marked  consistency  with  this  conclusion.  The  VI  meas¬ 
urements  (fixed  station)  are  generally  smaller  than  the  corresponding  value 

i 

for  the  OI  (mobile  station)  measurements.  Their  differences  are  recorded 
as:  dj  ~  fo  "  ^v  1  xo  “  y v 
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The  paired  comparison  tort.  E2,  f  igures  16,  17,  18.  1‘),  require  s  that 
two  random  samples  be  independent.  The  data  shows,  however,  that  u 
pair  of  measurements  for  VI  and  OI  for  any  particular  time  are  of 
approximately  the  same  magnitude.  In  other  words,  the  variance  within 
the  blocks  is  small  compared  to  the  variance  between  the  blooks. 

The  following  data  from  Figure  (9)  wore  taken  at  11:30  hours: 


Day 

*o 

h 

Ay  v 

Xr,-yv 

1 

10.80 

10.  80 

0 

2 

10.00  * 

0.80 

9.  80 

1.  00 

0.20 

3 

9.00 

1.00 

9.20 

• 

0.  60 

0.20 

4 

9.50 

0.  50 

9,50 

0.  30 

0 

5 

10.00 

0.50 

10.  00 

0,  50 

0 

6 

10.50 

0.50 

10.  50 

0.  50 

0 

7  , 

9.50 

1.00 

9.  60 

0.  90 

0.10 

8 

9.50 

0 

9.  50 

0.  10 

0 

9 

10.50 

1.00 

10.  40 

0.90 

0.10 

In  other  words  Ax0,  6yv  >  xQ-y  v;  as  a  result  of  the  homogeneity  within  the 
blocks,  the  new  experimental  design,  that  of  Paired  Differences,  utilizes 
the  nine  difference  measurements,  di,  per  time  elot  to  test  the  hypothesis: 
H0:  4v  “  Ml  vs.  Hi:  *  Ul 
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with  the  t  test  .-iuj  *  .  iii'ihence  intervals  3e  shown  in  equations  l 2  a,  13a, 


14,  23a.  This  statistical  design  is  a  simple  example  or'  a  randomized 
block  design.  The  test  is  commonly  called  a  paired  difference  test.  It  is 
emphasized  that  the  pairing  was  part  of  the  planning  of  the  experiments,  and 
was  not  done  after  the  data  were  collected.  Each  of  the  blocks  consists  of 
the  two  observations  xQ  and  yv  for  the  same  day  at  a  specific  time. 

(See  Figures  8,  9, 10,  <ind  11.) 

By  comparing  the  comp  ited  confidence  intervals  for  the  85  time  slots 
for  the  paired  difference  model  with  those  of  the  unpaired  model,  see 
Figures  14, 15, 18, 19  we  see  a  decided  gain  in  Information  favoring  the 
randomized  block  design.  The  tfain  of  Information  ii  reflected  in  the 
difference  in  the  width  of  the  confidence  intervals.  Again  using  data  at 
10:00  hours,  in  Figure  20,  the  interval  for  the  paired  comparleon  test 
Ipc-<-.  9405,  .  9293)  .  The  interval  for  the  paired  difference  test 
Ipd"(“>  1859,  .1981),  and  Ipd  <  Ipe  . 

The  IpH^  i«l.  ....  85  are  much  narrower  as  a  result  of  blocking  in  this 
experiment.  Figures  20,  21,  22,  and  23  show  the  comparison  of  the  confidence 
limits  for  both  methods,  as  well  as  a  large  reduction  in  the  standard  de¬ 
viation  Sh  as  compared  to  the  pooled  standard  deviation,  Sp.  of  the 
unpaired  observations.  Variances  are  presented  graphically  in  Figures 
24,25. 
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Vo  ;ij -priHuable  difurem-t’  exists  between  the  data  of  t1  >■  f/xt-d  and  m.diilt 
iono, sondes  for  a  distance  of  60  Km.  The  data  of  the  fixed  terminal  is 
very  nearly  Identical  to  those  of  the  mobile  terminal  for  this  distance.  This 
means  that  only  one  terminal  is  needed  at  tills  distance  to  provide  useful 
ionospheric  data  under  these  given  conditions.  The  result  bearH  out  the 
expectation.  Experimentation  in  planned  for  investigating  critical  fre¬ 
quencies  at.  distances  b« yonri  60  Km,  (up  to  500  Km),  to  determine  the 
distance  within  500  Kin  v>  nere  the  conclusion  becomes  invalid.  This  would 
provide  insight  as  to  an  extreme  distance  limit  for  the  usefulness  of  vertical 
incidence  ionospheric  soundings  with  respect  to  critical  frequency. 
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FIELD  TESTS  FOR  A  NEAR-REAL  TIMI 
IONOSPHERIC  FORCASTING  SCHEME 
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REDUCED  IONOSONDE  DATA 
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TiME  OF  DAY 


POSITION  OCATlo;.  VIA  MULTIPLE  TR  fAN'.'JLAT  ION* 

Glenn  A.  Stoops  and  Edward  I  <?j»i  tmitge!  ,  'r. 

Litton  Scientific  Support  Laboratory 
Fort  Ord,  California 


1.  iNThODliCTit).,'.  Classical  lr  iangulaL  ion  in  tin.  plane  involve:;  locating 
an  unknown  position  by  measuring  its  direction  trim  two  known  points  and 
finding  the  intersection  point  of  the  two  location  lines.  If,  nu*ee  generally, 
there  are  n  known  points  reporting  directions — and  there  are  error::  in  the 
observed  directions — then  the  n  lines  cannot  be  expected  to  intersect  in 

a  common  point.  Two  different  methods  of  obtaining  a  closed  form  estimate 
of  the  true  position,  with  variations  on  each,  will  he  derived  and  discussed, 
along  with  in  error  an;  lysis  .■£  each  method. 

2.  ESTIMATION  METHODS. 

a.  Least  Squares  (‘LSQ).  The  n  known  positions  are  denoted  by 
^xi'  ^i.) »  i  “  1«  2,  ....  n,  and  the  observed  directions  by  the  respective 
angles  1 ^ .  This  yields  an  equation  for  the  i  th  direction  line  L : 

y  -  yL  “  tan  <.  (x  -  x1). 

Tue  perpendicular  distance  d  from  an  arbitrary  point  P  (x,  y)  to  the  luu* 

L,,  is  given  by: 

d  (P,  Li)  «  |  (x  -  xx)  sin  -  (y  -  y^)  cos  <1^  |. 

The  LSQ  method,  roughly,  determines  a  point  that  is  close  to  all  the  lines 
Li,  in  the  least  squares  sense.  Specifically,  define  the  function 
f  (P)  =  f  (P  (x,  y)) 
n 

=  i:  (d  (p,  l  )r 
i=l 

-  d2  (P,  L  ) 

2  2  2  7 
=  ::i(x  -  xx)  sin  +  (y  -  y.)  cos“-ri 

-2  (x  -  xi)  (y  -  y^ )  sln;^  cos^J. 

The  (unweighted)  LSQ  estimate  is  the  point  P  that  minimizes  this  function. 

A  slightly  more  general  function  is 
g  (P)  -  Z^d2  (P,  L.), 

where  1*^}  is  a  set  of  fixed,  but  arbitrary,  nonnegative  numbers.  Physically, 
the  minimization  of  this  function  corresponds  to  weighting  some  of  the 


*This  article  has  been  reproduced  photographically  from  the  author’s  manuscript. 
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mfori  i  l  j  .m  mot--  .n-ivilv  than  the  rest.  Setting  «.*  -i  t !  ■  ■  .  equal  '  1  r  ■ 

i 

.  -v.  ,11  .>  ..  1  .nCt  igiitCu  i"  J:>C  ■ 

:  .1  „n  i  •:  g  (.P;  is  straightforward:  oo  'rule  g/.-j;  and  -g/  y  , 

^,-t  oil..  ■.  jiial  a.  and  solve  the  pair  of  t-ciiol  Ions .  Tin-  estimates  derivet 


(  <  ,x  sin  ;  ,• .  sxn  .  .  cos  .  / 

i  :  i.i  l  i 


.Mi  .cos:.-  .sin: .  cos :  .  .y .  cos- ; 

i  .  i  ii  i  i  r  i  i  i 


■:  .  .sin’':  . 

,  *  i 

■ ..sin;  .cos:  . 


■* -  •  .  s ini' .  cos  * ,  i 

.  L  a1  l\ 
cos*,*^  i 


o.  iV  int  ■  :  'intersection  (POI).  ['.act1  pair  of  direction  J  ines 

intersects;  ir.  a  isi-jnc  nnint,  and  there  are  (  ”  )  *  n  (n  -  l)/2  s,ueh  points. 

tiie  iiit.-fM.-t.liou  ■  ant  of  1  unis  l,  anti  l.,  1’..  v .  J ,  is.  derived  oa.siU 

i  J  t  j  l  j  i ) 

fro-  tiicir  equ.if  i-J'is  and  is  given  by: 

o.j .  .  c  a  cos  :  .  -_XjS in:  .)-  o.s:  .  (y^cus:  ^-x^sin.M  ) 
lJ  sin  (;-i  -  .(j)  ^ 

-.  >i :  .  ',_v  . cos  :  . y_x  .  sin;  )-gin:  ^  (y^cos-^-x^sln; 

‘  '  s  in  ( :  .  -  :■  j.) 

...  .’  .  .  ,.u.  j.  .- «li:-  :  v  ■  .i  mi, .ply  involves  computing  a  weighted  or 

:pu\.  ighte..  av.-r  ’1  t.-e  (.  ,  )  points-  I’...'  That  is,  a  set  of  nonnogitiv. 

1 J 


,  ■ .  I  -  i‘  ) 


-s  selected,  and  tiie  coordinates  of  tiie  estimated 


;  i  i  ;  ■  i  r  ■  ■ : 


I’Ol  1  .]  1J  1]  l  j  1.] 

■’pul  *  i  j  'i  j-’ij/i:  j  ‘  i  j  1 


i.iSicst  course  is  to  ciioos.-  the  weights  so  that  they  sum  to  1, 
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I:  one  assumes  that  tin-  errors 


i.  approximation  Ioi:d  in  i-irkuk  a.,alysi:n. 


in  measuring  the  true  unties  iron  Liu-  respective  known  points  i’ .  me 
small,  then  the  following  approximation  is  a  verv  sound  mn  ami  aids 
considerably  in  tlie  error  analyses.  Note  tiiat  tiie  error  in  locating  an 
unknown  position  is  in  general  a  function  o;  tin?  position,  so  that  in  this 
sense  tiie  errors  derived  are  conditional  errors,  conditioned  by  tin-  actual 
(unknovai)  position. 

Let  (iu,  '■  ^>  denote  the  true  distance  and  direction  of  tiie  unknown 

noint  1'  from  t:;e  i  cN  known  a...  it  P  .  ine  line  1.,  an-.,  Li.  rough  P.  .aid 
i-  i  i  i 

misses  P  by  a  distance  of  approximately  i; .  .  (see  iigure).  1  he 


geometry  of  tbe  error  analyses  is  eased  considerably  if  line  I. .  is  replaced 
by  line  parallel  to  tiie  line  tror.i  P^  to  P  and  displaced  by  an  amount 
K i i i  — ^  ;  in  the  appropriate  direction.  Lines  1,^,  and  .  are  virtually 

4 

indistinguishable,  and  both  miss  P  bv  the  same  amount. 

4.  LRROR  ANAhYSLa.  If  P  (x  ,  y  )  denotes  the  true  point  to  be  located 

o  o  o 

and  P  (x,  y)  denotes  the  computed  estimate  (via  either  i.SQ  or  POi),  then 

the  b  isii  i  rit.  in  -n  used  to  assess  the  <;ual  ity  of  tiie  respective  eat  Lina  lift 

>  •, 

is  L  |  (x  -  >.  +  i, v  -  y  )“1  =  I.  1  p  -  1*  -j,  tiie  mean  squared  radial  error 

i »  . *  'O' 

The  only  assumptions  made  about  the  random  variables  :  are  that  they  be 
independent,  unbiased,  and  have  common  variance  . 

a.  I.SQ  Method.  L'sing  Liu  counterc lochwise  convention  for  positive 
angles,  one  can  show  that  the  distance  from  an  arbitrary  point  P  (x,  y)  to 
is  given  by: 

d  (P,  A.)  =•  (x  -  x  )  sin--  .  -  (v  -  v  )  cos  -  .  +  R,  .  .  )  ,  . 

1  o  1  ■  -  O  1111 

liiven  P  — and  thus  -K,.  and  ,  — Liu-  point  P  (x,  ,  ),  because  ot 

o  i  i  1  l.bQ  1..-^  ’ 

the  way  iL  was  computed,  also  minimizes  the  [unction  h(P)  =  :...d“  (P,  . 


(conditionally)  in 


fli nil-lining  h  (P)  expresses  x}  gg  and  y^gg 


’  t  !:l  expressions  ore: 


i  Hi  a  )  .  sin.1  .  (’.os'.‘ .  ) 

1  j  i  i 


(}  '  iKi(  i”  i^3in°i^  ^£icos 


and 

tu:’ 


nlaiiy  tor  y 


ISO 


of  (: 


■i  j  2 

•’.sin*-1  (;.'ic..cos'‘-'.)^)  -  (Ei.  jSinO^cost'^) 

Note  inanediateiy  that  x^gg  and  y^gg  are  linear 


s). 


*  that  the  mean  squared  radial  error  is  in  fact  a 
t unction  only  of  the  variance  of  ($^-0^)(and  the  geometry). 

iru-  f rro;  xrress'ou,  K  [(x  -x  )2  +  (y,  -y  )^J,  is  still  relatively 

O  O 


.  i  ay  the  t.a  t  that  the  ran.: 
n  *  an  j  .  "  .  1  i  give;:  by : 

i  ■> 


'  •allies-  i  j  are  independent 


4  .  U.  .• 
1 


) 


+  I  (.  ■  j  s  tu 


.  .  i  os  - .  slab ,  ]  (Li, ,  ^R.  ^sin"ii . ) 

t  i  1  i  i  l 

^  +  (i.t^cosd^sindj^KEJl.^R.^cos^O  ) 


2  2 

-2  (.:.. .  t  )-l < .  sind . )  (If. , )  (21 .  R.  sinO  cosO.) 
t  i  t  l  ii  i  i 

"T" 


7T7T 


J  . 

-i  a  . 


l(::^sin  i*  )(i;jc.^cosfc0  )  -  (££  sinO  .cosO.)  ] 

llxl  111 

h.  I’Ol  Method.  T'e  coordinates  of  the  point  P^  are  determine* 
,  approx; natcly  by  finding  the  intersection  of  A^  and  Aj.  This  yields: 


0< .  ( :  j  -  ‘  .))cos  |"(Rj(*j"*,-j))c°s<)j_ 


f - 


sin  ( " i”'* j f 


U< .  ( .  i~-:  t )  )sin"j-(Rj  (>j-Oj))sin01 


yo  + 


i>  i  nci 

:  the  estl 

u\n  I  e 

UP<.r  y 

'"'Put 

Lr 

i ' i iAi  i 

VF’li 

i 

j  I.1  ’  i  j 

’  i  * 

• .  1  1 

l  i  .hi 

!-ro; 

•  in  (c-j-Oj) 

)  is  given  by 


i'-: J 


1, 


t  2 

+  (>l>01-y  )  1  derived  by  combining  the 

terms,  etc.,  .squaring,  and  careful  bookkeeping,  noting  again 

2 


tie  independence  and  common  variance  a  of  the  random  variables.  For 


+  x 


nience  of  notation,  define  t  **  i  ,  if  j ,  and  define  A. 


Mi*  “  . . .  ij 

•»  .-,11  i*j.  Then  t lie  mean  squared  radial  error  is  given  by: 


iJ 


t .,/sin  <e.-0.) , 


ij 


i  J' 
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•  R  ■  *  [  (  .  L.> .cos' ■.  +  (. .  >  ain't  .  )^] 

i=1  i  J?i  ij  ]  ij  J 

5.  IMPROVEMENTS  BY  ADJUSTING  THE  WEIGHTS.  The  derived  error  expressions 
are  too  complex  to  permit  many  general  observations  to  be  made.  Extensive 
study  of  examples  indicates  that  the  LSQ  method  leads  to  smaller  error 
than  does  the  POI  method.  In  particular,  it  is  conjectured  that 
unweighted  LSQ  is  always  better  than  unweighted  POI.  However,  either 
method  can  be  Considerably  Improved  through  the  use  oi  even  imperfect 
information  about  tha  unknown  location.  In  the  following  subsections 
idealized  weight::  art  J.-rived  tor  eacu  method,  weights  that  minimize  the 
respective  error  expressions  but  are  unattainable  because  they  require 
perfect  information  about  the  unknown  locations.  In  later  sections  these 
idealized  weights  are  interpreted  as  yielding  lower  bounds  on  the  error 
expressions,  bounds  that  cannot  be  attained  but  can  be  approached  by 
various  iterative  schemes. 

a.  LSQ  Method.  The  intention  here  is  to  find  the  set  of  nonnegativo 

weights  ■'>.,}  that  minimizes  the  (conditional)  error  expression,  li,  for  a  . 

particular  unknown  location  P  .  Of  course  the  set  is  different  for  each 
-  o 

P^  and  thus  cannot  be  derived,  even  .in  theory,  without  perfect  knowledge 

of  itself.  However,  the  mere  existence  of  such  a  minimal  set  indicates 

a  lower  limit  on  how  much  improvement  can  be  expected  even  with  partial 

information  about  P  . 

o 

Hole  first  that  E  is  homogeneous  in  the  that  is,  multiplying 

the  1^'s  by  a  common  factor  leaves  E  unchanged.  Note  also  that  a  minimum 
could  not  occur  along  a  boundary  (one  or  more  i^'s  equal  to  0),  since 
this  means  ignoring  some  of  the  data.  Thus,  a  necessary  condition  for  a 
locai  minimum  to  occur  is  that  all  the  partials,  be  equal  to  0  at 

some  point  (or  any  multiple  thereof).  One  solution  (and,  it  is  conjectured, 
the  unique  one)  is: 


R 


-2 


for  all  i. 


i  i 

Again,  this  solution  was  suggested  through  study  of  numerous  examples,  and 
it  can  be  chocked,  through  straightforward  but  tedious  computation,  that 
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it  does  indeed  satisfy  jE/3£^  *  0,  for  all  i.  The  simplicity  and  plausibility 
ot  t:iis  solution,  once  attained,  make  it  a  most  likely  candidate  for  unique 
global  minimum.  In  particular,  the  data  from  more  remote  points  P^  should 
obviously  be  weighted  less  heavily.  Interestingly,  the  minimal  weights 

_2 

do  not  depend  on  the  angles  {d  }.  The  error  expression,  E_.TM,  for  t  -  R,  , 

1  MIN  I  i 

is  given  by: 

::R1"2/((ZRi‘2cos20i)(i;Ri"2sin2oi)  -  (ER  ^cosi^sinO^2], 
a  relatively  simple  expression. 

b.  PC'T.  Method.  Since  ii  is  a  simple.-  expression  in  the  POI  i  ase, 
it  would  be  expected  that  minimization  is  also  easier  and  this  Is  true.  In 
fact,  since  E  Is  a  quadratic  function  of  the  2.  ^  ^  *  s  (or  A  ’a)  and  the 
constraint,  -  1,  Is  linear,  the  minimization  problem— via  partial 

differentiation  and  Lagr'ange  multipliers — reduces  to  solving  a  set  of 
simultaneous  linear  equations  in  2^.  once  more,  study  of  examples 
suggested  a  solution,  hence  the  solution,  which  is  given  by: 

■i;j  -  U1"2Rj"2sin2(0i-0j)/i^JR1"2Rj'2sin2(ui-0j), 

for  all  i<j.  As  noted,  the  optimal  weights  for  POI  involve  both  the  ll^'s 
and  the  t^'s,  perhaps  because  the  points  of  intersection  are  so  intimately 
tied  to  both.  The  no  si;  striking  fact  is  that  the  error  expression, 
it  pre .  i  sely  tne  same  as  that  derived  in  the  previous  subsection  for  hS(). 

(in  i.ut,  the  "optimal"  estimates  1 1  ictus  c 1 ves  are  identical  as  well.) 

Certainly  this  fact  is  more  than  coincidence,  and  Home  of  its  unifying 
implications  will  be  discussed  in  a  later  section. 

t).  i:.!  Lyi’di.i'ATluN  or  "UI’TiMAl."  WEIGHTINGS.  in  Section  5,  best  possible 

weigutings  were  derived  for  eacli  estimation  method,  given  a  particular  P  . 
as  stated,  these  weights  are  unattainable,  requiring  omniscience,  but  they 
indicate  directions  for  improvement  of  the  respective  unweighted  methods. 
Simply  stated,  some  information  about  the  location  of  is  better  than 
none  at  all.  This  suggests  that  an  initial  (unweighted)  estimate,  P^j,  be 
computed ,  via  either  method,  and  the  distances  R  ^  (and  angles  from 

each  to  be  determined.  Then  these  K^'s  (and  O^j's,  if  applicable) 

can  be  used  to  compute  a  second,  weighted  estimate,  P^j.  Since  P^j  is 
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s  can  be  used  to 


Naturally  tills  iterative  scheme 


presumed  better  than  ,  '*  «  associated  K.,'s  and 

( 1  ;  i 

compute  a  third,  weighted  estimate,  P^. 
can  be  continued  as  long  as  desired.  L'uioi  LuuUU  ly ,  very  little  work  has 
been  done  at  this  time  to  investigate  convergence,  and  rate  of  convergence, 
of  the  iterations  to  any  type  of  best  estimate,  but  Lliis  is  certainly  an 
area  for  continued  research.  it  is  felt  that  the  iteration.,  for  the  LSQ 
me tbod  probably  convoragc  rather  well,  whereas  the  i'ul  iterations,  because 
they  involve  angles  as  well  as  distances  and  the  data  is  in  terms  of  angles, 
uLlnw  the  possibility  of  circulr.rLty  and  instability. 

/.  CONCi.USlONS.  Two  methods  have  been  discussed  for  estimating  the 
location  of  an  unkmwi  point,  given  direction  data  from  n  known  points. 
Closed  form  tormulas  for  the  estimates  were  derived,  as  well  as  general 
expressions  for  the  muun  squared  radial  error  of  the  two  methods,  least 
squares  (LSQ)  and  point  of  intersection  (POX).  in  addition,  idealized 
"optimal"  weights  were  derived  for  each  method,  weights  that  reduce  the 
respective  errors  to  their  smallest  possible  values,  conditioned  by  the 

p 

true  location  of  the  unknown  point. 

It  was  noted  In  Section  b  that  tiie  optimally  weighted  solutions  for 
the  two  methods  are  identical.  To  understand  the  underlying  reason  tor 
tills,  consider  the  following  estimation  methods  for  an  arbitrary  point 


P  (x,  y),  let  v ^  be  the  angle  it  makes  with  P^  and  let  !•’  (P) 


i«.l 


<tV 


Kind,  if  possible,  file  point  P  that  minimizes  K  (P).  To 
the  degree  of  approximation  used  throughout  the  paper,  this  P  is  Identical 
to  the  optimal  LSQ  or  POI  point.  To  see  this,  rewrite  K  (P)  as: 


n  _2  2  v 

C’>  •  i  K  i’<,  (.'rr.)  J 

i-1  1 


•  •'jd  (P.  .’.j). 


where  }  is  the  optimal  set  of  weights  for  LSQ.  This  method,  finding  a 
least  squares  fit  to  the  raw  data  itself,  i;^i,  was  not  used  until  late  in 
the  investigation  and  still  is  not  preferable  to  the  others  merely  because 
it  does  not  lend  Itself  to  a  closed  form  solution.  Conceivably,  with 
sufficient  computational  facilities,  this  method  may  be  preferable  to  either 
LSQ  or  POI,  particularly  if  the*latter  require  a  large  number  of  iterations. 
Note  that  this  estimate  is  in  tact  the  maximum  likelihood  estimate  In  the 
case  where  the  ^'s  are  normally  distributed. 
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In  summary,  i.u  two  methods ,  LSQ  and  POI,  wore  presented  separately 
parallel  netmJi,  a Lthough  the  preceding  paragraph  does  tend  to  unify 
tu  two  uu  It."  eiie  ;d  theory .  Emphasized  was  the  simplicity  of  both 
.nethous,  especially  in  the  absence  of  unequal  weights.  The  POI  method 
Is  easier  to  visualize  graphically,  while  the  I.SQ  method  has  fewer 
computations  and  smaller  associated  error.  Both  methods  are  systematic 
and  easy  to  apply  in  many  practical  situations. 
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A  General  Computational  Algorithm  for  Bayesian  Confidence  Bounds 


by  Richard  l.'.  Clarke 

WATERVLIET  ARSENAL 
WATERVLIET,  NEW  YORK 


JNTRuDUCTION 


For  anyone  unfamiliar  with  Bayesian  analysis  this  paper  should 
serve  as  an  introduction  to  this  very  useful  confidencing  technique. 

The  aspect  of  this  paper  which  mignt  be  interestin'*  to  those  already 
familiar  with  this  subject  is  simply  that  I  have  outlined  a  computa¬ 
tional  algorithm  wl.icn  will  eliminate  the  very  messy  mapping  wnicn 
arises  in  applying  the  Bayes  formulation. 

I  found  it  convenient  in  what  follows  to  work  with  a  specific 
example  in  ordo-  to  make  a  few  basic  points.  A  more  general  treatment 
may  be  found  in  a  Watervliet  Arsenal  technical  report  by  the  same  title.1 

A  BAYESIAN  CONFIDENCE  BOUND  ON  RELIABILITY 

The  basic  intention  of  the  Bayesian  analysis  is  that  any  physical 
parameter  about  which  we  have  less  than  precise  knowledge  may  be  treated 
as  a  random  variable.  For  instance,  the  shape  and  location  Weibull  para¬ 
meters  might  be  treated  as  random  variables  if  we  are  using  the  Weibull 
density  to  represent  a  set  of  failure  data.  If  from  the  data  we  can  con¬ 
struct  the  joint  density  of  those  parameters,  then  rhe  reliability  den¬ 
sity  for  a  given  safe  life  or  the  safe  life  density  for  a  given  reliabi¬ 
lity  will  follow  directly. 

To  arrive  at  that  joint  parameter  density  we  must  first  specify  some 
prior  knowledge  of  those  parameters.  This  consists  of  stating  that, 
from  prior  testing  of  similar  items,  these  parameters  are  likely  to  be 
within  certain  bounds.  If  very  little  information  is  available  we  might 
say  only  that  a  certain  parameter  can  take  on  any  value  between  two 
limits,  and  that  each  value  between  those  bounds  is  equally  likely  before 
testing. 

Suppose  that  five  components  have  been  cycled,  under  actual  conditions 
of  fiel'd  use,  to  failure  and  that  these  failures  (cycles  or  hours,  etc.) 
aro  X^,  ,\-...Xv-.  From  prior  testing  of  similar  mechanical  components  we 
deduce  that  the  population  from  which  tnese  failures  come  can  be  reason¬ 
ably  approximated  by  the  two  parameter  Weibull  density.  Then  the  density 
of  X  is  given  by: 

exp[-(4:)f]  (|) 


This  article  has  been  reproduced  photographically  from  the  author's 
manuscript. 
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in  wruen  X  is  a  random  vanaoie  drawn  trow  w  jp,  ij .  for  any  set  of 
Weioull  parameters  we  have  that  the  joint  density  of  the  above  five 

independent  observations  is: 

g  (  X.  ,X,  X»/f  >Tj  =  f  f  (X;  /f,T)  (T) 

on 

g(X/&)  »  f  f(X;A*)  (3) 

Now  let  our  prior  knowledge  of  the  parameters  be  represented  by: 

jX**)  =  ,?,((*  ~)  oo 

In  this  case  we  might  say,  for  instance,  that: 

fiu  <  p  <  (*•  •«•&* 

T.  <  T<  T.  +  CSl*. 

(.5) 

E’UTEldHEfcii 


Bayes  theorem  then  states  that  the  posterior  knowledge  available  on  the 
parameter  space  for  these  five  observations  is: 


;.(•©*  /t) 


g’(YA»^,0) 
l  f(XA*)  p,('&<)Jl'e' 

•di  (6) 
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In  the  example  we've  been  following,  noting  that  gj  is  a  constant: 

^ _  (7) 

We  now  have  an  expression  which  assigns  a  probability  density  to  each 
point  in  the  parameter  space. 

At  any  point  in  that  space  we  can  related  reliability  (R)  to  safe 
life  (Xs). 


R  =  i-  ^  f(x/*»)Jtx 


or  for  the  Weibull  example: 


ex? 


ten 


Tnen  for  a  given  safe  life  the  density  of  the  reliability  estimator  may 
be  found  by  mapping  the  parameter  joint  density  (gj)  onto  reliability 
(R)  through  Eqn.  (8).  Analytically: 


r(1«-  * 


In  wnich  the  Jacobian  is  evaluated  from  Eqn.  (8).  A  one  sided,  (1  -cX) 
10(J%  lower  confidence  bound  on  reliability  is  then  the  100o<th  percen¬ 
tile  in  the  reliability  estimator  density  or: 


o<  — 
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Expression  fid)  is  not  particularly  simple  to  evaluate.  If  we  hap¬ 
pened  to  be  working  with  a  three  parameter  density  the  Jacobisi 
would  contain  three  terms  instead  of  two  and  two  of  the  variables 
would  nave  to  be  eliminated  by  integration  instead  of  one.  In 
general,  that  is  for  most  two  and  three  parameter  densities,  the 
integrations  could  not  be  carried  out  id  closed  form  and  some  nume¬ 
rical  or  computer  solution  would  be  required. 

A  SUBSTITUTE  FOR  THE  ANALYTICAL  MAPPING 

Instead  of  the  usual  analytical  mapping  as  defined  by  Expres¬ 
sion  (10),  we  can  start  directly  with  the  posterior  joint  parameter 
density  and  do  n  ntaaerical  mapping  onto  reliability  as  follows. 

In  inecifyi  g  tne  prior  parameter  density  (g^)  choose  a  rectan¬ 
gular  region  of  ■  ■innitian  such  as  in  £qn.  (5)  above,  then  divide 
that  regie'll  into  small  subregions  by  dividing  the  parameter  axes  into 
equal  intervals.  In  the  Meibull  example  we  have  been  following  the 
midpoint  of  a  specific  subregion  would  be  represented  by: 

^  U-  M  («-) 

Ti  T“+  jj-'iV-NT  o») 

In  which  Oi  and  Q2  represent  the  ranges  over  which  the  parameters  f 
and  T  are  defined  (See  Eqn.  (5)),  and  Np  and  Nj  are  the  number  of 
intervals  into  which  those  ranges  have  been  partitioned.  (See 
Figure  I) 


If  these  subregions  are  "small"  enough  (how  small  will  be  discussed 

)  ♦■Via  4n4r«+  nn  rf  4  am  f.  \  M*  *L«  » .  .U  — .i  I :  .  .  ..it* 

•*•*  */  I  'iV.l  •*  +.  tr  J  \.K  £  J  bHV  uo  i  W  ^  A.UII  Iliupuill  l  Ml  1  l 

be  a  reasonable  approximation  throughout  the  subregion.  Then  the  proba¬ 
bility  that  any  subregion  contains  the  actual  population  parameters  can 
be  represented  by 

=  Ok) 


Q,  Qi 

Nf  Mt 

This  probability  (I’ij)  can  then  be  associated  with  an  interval  on  the 
range  of  possible  reliabilities  by  calculating  the  reliability  for  tht 
parameters  (J  i  and  ij: 


An  actual  mapping  of  the  ijth  subregion  onto  reliability  might  look 
like  Figure  II  below. 
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*  "ill  approx  in.  ate  this  napping  by  dividing  the  reliability  axis  into 
interval*  and  assigning  the  entire  Pij  to  the  interval  in  which  Rih 
f«lis  Mathematically,  calculate: 


n 


with  ■  Maximum  reliability  possible 
fiMiN  *  Minimim  reliability  possible 

M  -  Number  of  intervals  on  the  reliability  axis 

then : 

_  "Rij  -TklU 
A'R 

Tnncating  I*j  to  an  integer  value  then  defines  the  interval  number  to 
which  Pij  is  to  be  assigned. 


By  running  through  all  the  (0i,  Tj)  combinations  and  assigning 
each  Pij  to  an  interval  on  the  reliability  axis  we  are  constructing 
a  histogram  which  approximates  the  reliability  estimator  density.  (See 
Figure  III) 


Figure  nr 


202 


The  accuracy  of  the  process  depends  only  on  the  interval  sizes  chosen. 
We  have  simply  replaced  the  integral  evaluation  of  the  mapping  process 
(Eqn.  10),  by  a  much  more  straightforward  numerical  evaluation.  Confi- 
denced  reliability  follows  from  the  histogram  by  replacing  Eqn.  (11) 
with  a  summation. 

Looking  back  on  the  process  we  can  note  that  certain  simpl ications 
are  possible.  The  evaluation  of  the  posterior  parameter  joint  density 
could  be  written: 

)  =  K  g  (*e*)  0*0 

in  which  the  constant  is: 


I 


But  the  sum  of  all  Pij  should  be  unity  so  that: 


In  other  words  we  do  not  have  to  evaluate  the  integral  in  Eqn.  (20). 

GRID  SIZE 

The  remaining  problem  then  is  to  determine  in  any  case  what  interval 
size  is  sufficiently  "small."  No  satisfactory  solution  to  this  problem 
is  presently  available.  In  applying  the  technique  to  actual  data  sets, 
nowcver,  the  following  points  were  noted. 

1.  Oie  specific  application  to  the  lognormal  density  with  a  uni¬ 
form  prior  parameter  density  yielded  the  following  confidence  bounds  on 
safe  life  for  a  given  reliability: 
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Grid  Size  Bound 
(Nx  X  Ha- )  (Cycles) 

15  x  15  800 

40  x  40  1150 

70  x  70  11S0 

In  this  case  15  x  IS  was  too  coarse,  but  40  x  40  was  agood  as  70  x  70. 

2.  An  application  of  the  three  parameter  Weibull  seemed  to  con¬ 
verge  with  a  grid  size  of  5  x  5  x  S.  Finer  grids  resulted  in  a  negli¬ 
gible  change  in  the  confidence  bound. 

3.  For  very  high  reliabilities  (.999,  .9999,  etc.)  the  lower 
confidence  bound  on  sife  life  seems  to  increase  as  the  grid  is  made 
finer.  This  would  indicate  that  this  method  yields,  for  given  grid 
size,  a  confidence  bouid  which  is  on  the  conservative  side  of  the 
"exact"  Bayesian  confidence  bound. 

CONCLUSION 

The  point  which  makes  this  computational  mapping  extremely  inter¬ 
esting  is  that  it  can  be  extended  to  any  distributional  form;  it  can 
be  extended  to  system  reliability  work  in  which  thfi  joint  posterior 
parameter  space  for  all  components  is  napped  onto  system  reliability* 
and  so  on.  It's  drawback,  of  course,  is  that  it  is  completely  computer 
dependent  and  for  In'*.;  -Trimeter  spac-.-^  the  computations  can  be  expen¬ 
sive. 
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Exact  Lower  Confidence  Limits  on  Normal  and  Lognormal  Reliability 

by  Royce  W.  Soanes,  Jr. 

WATERVLIET  ARSENAL 
WATERVLIET,  NEW  YORK 

This  paper  is  a  synopsis  of  References  (8)  and  (9)  which  were 
written  in  order  to  document  more  fully  the  solution  to  the  problem 
of  concern: 

Given  a  population  having  a  normal  or  lognormal  life  distribution, 
-nd  a  representative  sample  of  failures  drawn  from  this  population, 
calculate  an  exact  100  C  %  lower  confidence  limit  on  population  relia¬ 
bility  (R)  for  a  given  mission  life  (or  calculate  the  mission  life  (x) 
corresponding  to  a  given  lower  confidence  limit  on  reliability.) 

The  normal  reliability  estimator*  is  given  by 


(R* >  -I 


* 


By  performing  a  bivariate  change  of  variable,  the  joint  density  ofAR  and 
fir  may  be  obtained  in  terms  of  the  joint  density  of  and  CT 


* 


The  joint  density  of and  O  may  be  determined  from  the  fact  that: 

A  A 

(1)  yOC  and  O  are  independent  random  variables 

A  X 

(2)  'V  ~  has  a  chi-square  distribution  with  n-1  degrees 

CT*’  of  freedom  and 

A 

(3)  is  normally  distributed  with  mean yCC  and  standard  deviation 


♦Estimates  are  maximum  likelihood 

This  article  has  been  reproduced  photographically  from  author's  manuscript. 
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A  n 

The  joint  density  of  yLL  “d  0~  is  therefore : 

J>/*  A)  -  i  £  y  -  r\  *  t  W  i  or  J  J 

/(*■*)  ~pU  e 


The  joint  density 


•  a 

ity  of  /V  and  CT  is  therefore: 


h  (ft,?)* 


r  r-Frr  ‘hi 

s  uz  (t)  (z)  p  L 

'  rm w 


but  by  definition, 


/**  r  •*  *a’l 


/ri— I 


letting 


A,_  ^  eg) 1 
rr~) 


f  (?)  e 


*0^1- «J]  *  *  S 
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A  A 

Now  (Y  is  integrated  out  to  obtain  the  density  of  R: 


Since  s  is  a  dummy  variable  of  integration  and  R  is  the  argument  of  h, 
the  only  numbers  upon  which  the  form  of  h  is  dependent  are  R  and  n, 

The  density  of  the  reliability  estimator  is  therefore  a  one  parameter  (R) 
density  which  is  independent  of  the  life  density  population  parameters 
and  mission  life. 

Changing  the  argument  of  n  to  avoid  confusion  and  adding  tne  sub¬ 
script  R  to  n  t£  indicate  its  dependence  on  the  population  reliability  R, 
the  density  of  R  is: 
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The  distribution  function  of  R  is  therefore  given  by: 


U 


The  meaning  of  the  rieyman  method  of  finding  a  one  sided  confidence 
interval  for  R  may  be  explained  through  the  following  diagram: 


"Pu  >.ur.?  1  r-i  determined  by: 


/(?i<  r ,  ,<) 


HZ- ')  -  C 

i,e.,  1  is  determined  such  that  for  any  population  reliability  R,  the 
reliability  estimator  k  falls  below  1  lJO  C  \  of  the  time.  Suppose 
.ion  tnat  the  true  value  of  population  reliability  is  R  as  shown  in  the 
fxtniru,  we  don’t;  know  l<  or  r  but  we  do  know  1.  If  tne  experiment  is 
now  performed  and  the  reliability  estimate  R*  is  calculated,  the 
100  C  \  lower  confidence  limit  on  R  is  Rc  from  the  diagram.  This  is  so 
uecause  if  tne  experiment  is  performed  many  times,  R*  will  be  below  r 
100  C  \  of  the  time  and  hence  Rg  will  be  below  R  100  C  \  of  the  time. 

Confidenced  reliability  Rc  is  therefore  determined  by  solving 
for  1^: 


-  c 

f’J) 


A 

Before  this  is  done,  however,  the  distribution  function  of  R  should  be 
simplified,  ihanging  the  order  of  integration  in  Eqn.  (8)  and  making 
some  appropriate  changes  of  variable,  one  has: 
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*'  at  W  » 


l  *  V  UUt  1  > 


•  in  /  at  iu  cvtt;uabiu|(  cut  m  ic^t  ai  f  uiiv  nan. 


¥'>- 1 


> 


(12) 


Using  Eqn.  (12),  Eqn.  (lo)  now  becomes: 


If  R*  were  calculated  using  the  desired  mission  life  and  the  sample 
parameter  estimates,  Eqn.  (13)  could  be 'Solved  numerically  fbr  Rc, 
but  for  purposes  of  calculating  tables,  it  is  better  to  stipulate  Rc 
,  amd  solve  Eqn.  (13)  for  zR*  Instead.  This  was  done  for  confidence 
levels  of  90%  and  95%,  confideneed  reliabilities  of  .999,  .995,  .99, 
.975,  .95,  .925,  .90..S7S,  .85  and  ^aaple  sizes  of  2— 10,  15,  20.  25, 

The  equations  used  with  the  tables  to  calculate  mission  life  for 
the  normal  and  lognormal  models  are: 


(14) 


(IS) 
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REAL-TIME  SIMULATION  TECHNIQUE  FOR  EVALUATING  A  C-vpn-CFFKFR  ASSEMBLY 

Elwood  D.  Baas 

Systems  Analysis  Directorate 
White  Sands  Missile  Range,  New  Mexico 

f 

ABSTRACT .  Missile  simulations  of  systems  including  a  gyro-seeker 
guidance  assembly  have  often  excluded  the  gyro-seeker  representation  by 
assuming  that  some  ideal  proportional  tracking  ratio  will  be  achieved. 
Thus,  some  of  the  basic  characteristics  of  the  guidance  loop  are  omitted 
or  approximated.  This  paper  develops  a  real-time  simulation  technique 
so  aa  to  include  the  basic  functions  of  the  gyro-seeker  assembly  such 
as  precession,  nutrtion,  drift,  gain,  noise,  etc. 

In  the  first  etctlon  of  the  paper,  the  model  equations  are  derived 
and  are  used  in  a  discussion  of  the  system  parameters  and  system  dynamics. 

The  second  section  of  the  paper  presents  the  analog  computer  / 
mechanization  and  results  of  the  simulation,  some  of  which  have  beep' 
verified  by  system  experiments ,  and  some  predicted  by  analytical  theory. 

/ 

SYSTEM  DESCRIPTION.  The  basic  gyro-seeker  unit  consists  0/  a 
gyroscope,  rotating  gyro  magnet,  and  stationary  induction  colls  about 
the  gyro.  (See  Figure  1.)  Target  source  energy  is  collected  and  focused 
v,  to  produce  a  spot  Image  on  a  reticle  centered  on  the  spin  axis.  Wien  the 
'image  spot  la  off  center  the  reticle  pattern  produces  an  error  signal 
which' ia  modulated  at  spin  frequency.  The  amplitude  of  the  error  signal  . 
is  a  function  of  the  radial  displacement  of  the  image  from  the  reticle 
center,  while  the  phase  corresponds  to  azimuthal  position  about  the 
seeker  axis.  After  being  amplified  and  filtered,  the  signal  is  fed  to 
the  precession  coll  which  torques  the  gyro  magnet  so  as  to  preceas  the 
gyro  toward  a  null  position  with,  respect  to  the  line  of  sight.  The  v 
processed  a.c.  signal  can  also  be  demodulated  into  orthogonal  components 
using  reference  coils.  The  demodulated  d.c.  components  can  be  used  for 
tracking  or  guidance  signals. 

J '  ACKNOWLEDGMENT .  The  author  wishes  to  thank  George  F>.  Hoffman, 

Karl  G.  Goodloe,  and  Nancy  M.  Wade  for  their  assistance  in  the  preparation 
of  this  paper. 

♦ 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  author’s  manuscript, 


215 


Mathematical  Model 


Two  coordinate  systems  will  be  referenced  in  the  model 
derivation.  The  g>  system,  or  ground  system,  will  be 
fixed  at  the  initial  gyro  position  with  gj  axis  horizontal 
and  pointed  at  the  initial  target  ground  position.  The 
s>  coordinate  system,  or  seeker  system,  will  be  fixed  to 
the  center  of  gravity  of  the  gyro  assembly.  The  si  axis 
is  along  the  gyro  spin  axis  and  ?2  is  along  the  North-South 
axis  of  the  gyro  magnet.  (See  Fig.  2). 


TARGET  A 


The  yaw,  pitch,  rcii  sequence  for  the  Euler  transformation 
between  coordinate  systems  is  given  by, 

s>  -  M  (fi,  $2,  *3)  g>>  where  the  subscript  denotes 
rotation  about  the  respective  axis.  In  detail  then,  the 
model  Euler  equations  are: 


'sA 

r 

s2 

■ 

■ 

*♦3 


♦3  ■  sec  $2  (“2  sin  $1  +  u>3  cos  +1) 


C$2 

S*2  S$X  +  C$3  C$i 
S$2  C$i  "C$3  S$i 


-•♦a 

C$2  S$i 

C$2  C$! 


/sA 

82 

W 


$2  “  u>2  cos  $1  "  1D3  sin  $1 
•  • 

$1  ■  mi  +  sin  $2  $3 

$3  ■  ($3)0  + 

$2  “  ($2)0  + 

$1  "  ($l)0  + 

To  derive  an  equation  for  the  seeker  output  signal  we  assume 
that  the  reticle  is  parallel  to  the  s2  -  s3  plane  and  centered 
on  the  axis.  The  optics  produce  a  target  imege  point  on  the 
reticle,  as  shown  in  Fig.  3,  whenever  the  ij  axis  deviates  from 
the  line  of  sight.  The  radial  displacement  d  of  the  image 
point  from  the  reticle  center  is  proportional  to  the  angle  e 
between  the  si  axis  and  the  line  of  sight  vector  to  the  target. 
The  distance  d  is  now  modeled  by  deriving  a  distance  r  in  the 
?2  ~  s3  plane  which  is  proportional  to  d.  Let  0  be  the  unit 
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vector  along  line  of  sight  vector  in  the  ground  coordinate 
system.  Then, 


a 


+  Y  g2  +  Z  g3,  where 
K  E 


Let  og2  and  denote  the  components  of  o'  in  the  seeker  coordinate 

system.  (See  Fig. 4  ).  Then  r  ■  /o'*  '  +  depends  on  c  and 

s2  s3 


Figure  4 


not  on  the  length  of  o’.  Also  the  length  r  is  proportional 

to  the  actual  displacement  d  which  depends  on  the  optics 

and  other  physical  parameters.  The  azirruthal  phase  determined 

by  the  angle  e  measured  from  the  J2  axis  to  the  image  .point 

line  is  given  by  sin  e  ■  °sa  .  (See  Fig. 4  ).  This  derivation 

r 

assumes  that  reticle  modulated  output  has  the  form  of  a 
sine  wave.  Other  wave  shapes  could  be  generated  by  using 

l 

various  reticle  patterns  and  electronic  processing.  For  a 

'  1 

discussion  of  other  wave  forms  and  their  effect  on  gyro 

1 

precession  see  reference  1.  Then  eQ  ■  Kr  sin  e  ■  Kc^is 
an  equation  which  represents  the  seeker  output  signal.  To 
compute  oe  we  recall  that, 

s3 

os  ■  M  (*!,  ‘.M,  $3)  Og,  so  that 

0  -  X  [0*3  S*2  ♦  s*3  S*j]  ♦  Y  [s*3  St2  Ctl  -  c<>3  s^] 

S3  R  R 

+  Z  C<?2  C$i  . 

R 

The  signal  eQ  ■  JCr  sine  is  amplified  and  applied  to  the  coll 
about  the  gyro  whose  field  acts  on  the  permanent  gyro  magnet 
to  process  the  gyro.  The  variation  of  the  magnetic  field  is 
thus  proportional  to  r  sin  d  and  its  direction  is  perpendicular 
to  the  plane  of  the  coil  as  shown  in  Fig.  S  .  Suppose  that 
the  magnetic  field  fc  of  the  coil  makes  an  angle  X  with 
respect  to  sj,  then,  |B_  .  sj  |  ■  |Bj  cos  X  5j.  The 

magnetic  field  of  the  permanent  magnet  can  be  written  simply 
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PRECESSION  COIL 


as  ffn  -  |*ys2.  The  interaction  of  the  two  magnetic  fields 
produces  a  torque  which  tends  to  align  the  two  fields.  This 
torque  is  given  by  the  vector  cross  product, 

T3S3  ■  | B  |  s2  x  |5 _|  cos  x  i'i  where  |S _|  -  Kr  sin  0  and 
|I  |  is  constant.  Note  that  the  torque  T3  will  be. a  maximum 
when  the  spot  line  is  along  the  S3  axis,  then  the  rotation  of 
the  magnet  will  be  in  the  s2  -  plane  and  the  precession  in 

i  . 

the  s3  -  s,  plane.  Let  T  ■  Ti?i  +  T2s2  +  T3S3  be  the  total 
external  torque  acting  on  the  gyro.  Then  the  angular 
accelerations  are  given  by, 

“3  10  Ti  ‘  tt,lu2  ih  *  U 
I3  I3 

w2  “  Xl  '  ,*'lw3  til  ‘  ill 
h  l2 

«1  -  IL  -  u)3ta>2  Iil_L_U 
*1  M 
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If  we  assume  that  the  only  torque  acting  is  T3  as  derived 

above,  and  that  the  transverse  moments  are  equal,  then 

the  equations  reduce  to: 

t»3  "  Tj.  ‘  [I2  “  Ill 
I3  I3 

o)2  ■  -  wi<»)3  [ 1 1  -  I3] 

12 

Integrating  the  components  of  Z  yields  the  angular  velocity 
components  of  Z  of  the  seeker  relative  to  the  seeker  coordinate 
system.  Using  the  seeker  to  ground  transformation,  uf  can 
be  transformed  to  Wg  ■  $1  +  $2  +  $3  by, 

*  sec  $2  (“>2  sin  $i  +  “3  ^  ti) 

• 

$2  11  i*>2  cos  *  U3  sin 
•  • 

4>j  "un  +  sin^^s* 

Integrating  i>  yields  the  angles  ♦»  of  the  ground  coordinate 
system  which  are  needed  to  compute  es-  Thus  the  loop  is 
closed. 

Orthagonal  components  of  the  seeker  signal  are  obtained  by 
demodulating  the  signal  using  two  reference  signals.  Reference 
signals  can  be  obtained  by  mounting  coils  about  the  gyro  90* 
a pan .  (See  Fig.  6).  A  sinusoidal  voltage  is  produced  by  each 

I 

coil  nr.  the  flux  lines  of  the  gyromagnet  cut  the  windings 
uf  the  coil.  These  signals  can  sinply  be  modeled  as  Ksin  (*i+y) 
and  Kcos  ($i+y)  where  y  is  the  angle  the  coils  are  rotated 
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Figure  6 


from  the  ^  reference.  The  inputs  to  each  demodulator  are 
the  seeker  signal  and  one  of  the  reference  signals.  The 
filtored  output  is  a  d.c.  level  which  is  proportional  to  the 
error  amplitude  component  in  the  respective  plane  or  direction 
determined  by  the  angle  y.  The  mathematic  1':  representation 
for  tho  phase  demodulator  is  not  given  since  the  actual 
electronic  network  is  easily  adapted  to  the  analog  computer 
components.  Fig. 7  shows  a  typical  phase  demodulator  bridge 
network  used  for  one  plane. 

Tho  basic  gyro-seeker  model  which  has  been  developed  can 

I 

easily  be  expanded  or  modified  to  include  hardware  changes 
or  known  parameter  variations.  For  example,  the  actual 
seeker  output  signal  could  be  a  function  of  source  intensity, 
target  range,  noise,  filtering  and  other  phase  and  amplitude 
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Figure  7 
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transfer  function  additions  or  nultiples  of  the  signal 
eQ  ■  Kr  sin  e. 


« 
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Simulation 


The  simulation  of  the  gyro-seeker  employs  the  mathematical 
equations  as  derived  in  the  math  model  with  the  following 
exceptions: 

1.  It  was  assumed  that  the  gyro  speed  was  constant. 

This  is  the  case  in  a  gyro  when  the  motor-driving  torque  just 
balances  the  fr.  ction  torques  so  the  gyro  spins  at  a  constant 
rate.  Thus  sin  and  cos  were  obtained  by  running  in 

oscillator  at  the  required  spin  frequency. 

i  /.  ■ 

2.  The  equation  <f>i  ■  +  sin  <j>2  $3  was  approximated  by 

/  •  /  * 
the  equation,  ■  wj ,  i.e.,  it  was  assumed  that  sin  4 2  $3 

/ 

The  computer  mechanization  diagrams  are  presented  on  pages  13 
to  IB.  /Since  this  particular  mechanization  is  part  of  a  hybrid 
missile  simulation,  some  of  the  confutations  are  shown  as 
digital.  It  should  be  clear  to  the  reader  how  on  all-analog 
simulation  could  be  obtained  from  the  given  mechanization. 

The  parameters  such  as  spin  frequency,  moments  of  inertia  and 
loop  gains  were  obtained  from  experimental  data  taken  from  the 
seeker  hardware.  After  all  parameters  were  obtained,  the 
simulation  was  verified  by  comparing  the  response  characteristics 
of  the  hardware  with  those  of  the  simulation. 


226 


R 


GfflO  NUTATION  OSCILLATOR 


230 


i_— ■- 


in 


J — l 
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Some  simulation  recordings  and  description  of  results  are 
presented  on  the  following  pages. 

Pages  21  and  22  show  the  response  of  the  gyro  alone  to  a 
constant  amplitude  sinusoidal  torque  at  spin  frequency  applied 
about  the  g3  axis  and  phased  so  as  to  precess  the  gyro  in 
the  *3  plane.  The  difference  in  the  two  recordings  is  the 
result  of  a  change  in  the  moment  of  inertia  ratio  p  *  1 3/I  i 
whicn  can  be  seen  as  a  change  in  the  nutation  frequency  on 
the  uz  anc  ..-j  channels.  The  frequency  of  the  nutation  is 
determined  by  w;  (1  -  p)  while  the  amplitude  depends  on  the 
initial  conditions  (w2)0  and  (ui3)0 .  (See,  e.g.,  reference  3 
for  an  analytical  derivation) .  The  gyro  precession  which 
is  seen  as  a  change  in  $3  is  proportional  to  the  amplitude  of 
the  applied  torque. 

Page  23  shows  the  closed  loop  response  for  a  given  gain  Ki 
as  a  multiple  of  the  feedback  signal  r  sin  9 .  In  this  case 
the  seeker  was  not  tracking  (as  can  be  seen  by  4>z  and  $3), 
but  was  locked  to  a  stationary  target,  X  •  C,  Y  ■  0,  Z-0. 
Thus  r  sin  e  has  a  small  amplitude  and  a  phase  which  is 
changing  rapidly  to  compensate  for  directional  changes  of 
the  spin  axis  from  the  line  of  sight.  Page  24  shows  the 
response  to  a  target  moving  at  a  constant  rate  in  the  Y 
direction.  This  condition  results  in  an  error  signal 
Ki  r  sin  8  which  has  a  constant  amplitude,  reflecting  the 
constant  target  rate,  and  a  fixed  phase  dictated  by  the 
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Y  direction.  (Note  that  since  the  torque  applied  to  the 
gyro  is  aoout  the  S3  axis,  it  appears  on  the  *>3  channel.) 

The  demodulated  outputs  of  the  error  signal  for  target 
rates  in  the  Y  and  Z  direction  are  shown  on  pages  2  5  and 
26  respectively.  In  this  case  y  was  chosen  as  0°,  so 
that  r  sin  e  was  in  phase  with  one  reference  signal  and  90* 
out  of  ph&so  with  the  other  reference  signal,  for  each 
tracking  con  lit  ion.  Thus  the  perpendicular  tracking 
directions  result  in  alternate  full  value  and  zero  value 
readings  on  the  demodulated  outputs  Rj  and  R2  as  shown  on 
1  pages  25  and  26  .  The  Slow  rise  of  the  Rj  and  R2 
|  signals  is  due  to  filtering  on  the  demodulated  outputs 

!  and  not  to  the  demodulator  circuits. 

! 

i  Error  signals  can  also  be  produced  by  gyro  motion  such  as 
;  gyro  drift.  If  we  assume  a  constant  gyro  drift  rate  and 
stationary  target  conditions  then  an  error  signal  is  produced 
to  overcome  the  drift.  Page  27  shows  the  simulation  results 
for  these  coalitions.  In  this  case  drift  was  produced  by  an 
appropriate  torque  in  the  1 3  plane.  The  result  of  the  error 
signal,  fcr  sin  9,  generated  by  these  conditions  can  best  be 
observed  on  the  demodulated  outputs.  One  can  think  of  these 
outputs  as  false  tracking  commands  caused  by  gyro  drift. 
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CHARACTERISTICS  OF  A  SIX-COMPONENT  ROCKET  ENGINE  TEST  STAND 

Aubrey  W.  Presson 
U.  S.  Army  Missile  Commend 
Redstone  Arsenal,  Alabama 


INTRODUCTION.  LANCE  Developmental  testing  vithin  the  T4RE  Laboratory 
has  been  accomplished  on  test  stands  designed  to  measure  the  six  components 
of  thrust  reaction.  The  basic  problem  inherent  in  such  stands  is  that  of 
restraining  the  engine  with  a  measurement  system  that  permits  the  engine 
six  degrees  of  freedom,  without  the  introduction  of  unknown  effects  upon 
the  engine.  This,  if  course,  la  further  compounded  by  the  requirement  to 
supply  propellants  ihru  a  high  pressure  plumbing  system  that  shunts  the 
measurement  system.  It  is  obvious  that  a  thorough  stand  calibration 
program  must  be  implemented  to  resolve  this  problem. 

The  facility  requirements  stated  for  the  present  phase  of  LANCE  are 
presented  in  Table  I.  Paralleling  these  are  our  estimates  of  our  then 
existing  capability.  These  estimates  were  derived  from  extensive  calibra¬ 
tion  teats  performed  on  the  original  test  stand.  Only  a  casual  observance 
is  required  to  realize  that  this  represents  a  significant  stap  forward.  A 
lass  casual  but  limited  preliminary  error  analysis  indicated  mprai  clearly 
the  difficulties  involved  and  concluded  that  a  portion  of  the  req|uiremeht 
was  clearly  beyond  the  state-of-thc  art.  Briefly  this  is  indicated ,  when 
load  cell  accuracy  requirements  are  derived  from  consideration  of  the 
angularity  and  position  rsquireswnts.  Considering  only  the  load  Call 
capacities,  dictated  by  the  fhrust  magnitude  amf  the  practical  geometry  of 
the  stand,  the  position  requirement  means  a  vertical  load  cell  resolution 
of  0.05Z  F.5.  and  the  angularity  requirement  a  side  load  call  resolution 
of  0.3Z  P.S.  The  thrust  and  side  force  requirements  are  much  less  severs  - 
approximately  by  a  factor  of  3  times.  < 

This  dilemma  was  resolved  by  a  joint  decision  to  proceed  on  a  best 
efforts  basis.  It  is  this  effort  that  I  will  summarize  this  afternoon. 

II.  PRINCIPAL  DESIGN  FEATURES.  The  present  Thrust  Measurement 
System,  (Figure  1)  like  its  predecessor  system,  contains  seven  load  cells. 
This  is  the  TP  series  stand  with  component  convention  illustrated.  The 
four  vertical  cells  are  parallel  and  symmetrically  placed  about  the  vertical 
centerline  or  Z  axle.  These  cells  react  thrust  as  well  as  the  moments  about 
the  two  horizontal  axis  designsted  MX  and  MY.  Side  forces  designated  X  and 
Y  are  reacted  by  the  horizontal  cells  placed  on  or  parallel  to  these  axes.  , 
The  pair  of  cells  parallsling  the  Y  axis  slso  rssets  the  roll  moment  (MZ) .  * 
Each  load  cell  is  assembled  with  flexures  to  permit  maximum  compliance 
with  all  modee  of  loading  except  those  acting  directly  along  the  cell  axis. 
Thus,  the  magnitudes  of  force  interaction  inherent  in  the  system  are 
minimized. 
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Figure  2  illustrate*  additional  feature*..  These  view*  are  of 
the  present  system.  Features  to  be  noted  are: 

1.  The  unsupported  long  elbow  sections  of  pipe  in  the  propellant 
line  approaches  to  the  engine. 

2.  Arrangement  of  the  three  lines  approaching  the  encine  ■' 
symmetrically  about  the  vertical  centerline  in  an  attempt 
to  balance  the  restraints. 

3.  Displacement  of  the  plumbing  from  the  centerline  to  permit 
application  of  single  point  calibration  loads. 

4.  Inclusion  of  rupture  disc  housings  (appearing  aa  boxes 
between  th<  lines  and  engine)  that  permit  the  pretest 
lnstallatlci  of  these  discs  without  disturbing  the  pro- 
pollan*  line  connections  to  the  measurement  system. 

Load  cell  placement  and  alignment  of  the  system  and  especially  the 
alignment  of  the  calibration  input  devices  are  of  critical  concern.  Optical 
tooling  was  used  to  control  these  factors  to  precisions  of  better  than 
.005  inches  in  position  and  .05  mil  radians  in  orientation. 

The  basic  calibration  scheme  developed  for  the  original  stand  was 
first  employed  in  the  calibration  of  this  stand.  The  premise  of  this 
calibration  attempt  waa  that  there  are  many  sources  of  Interaction  which 
combine  to  produce  the  net  effect  on  the  system.  These  include  each  of 
the  components  of  force  input,  the  static  pressurlsatlon  of  the  propellant 
lines,  the  dynamics  of  flow  through  thase  lines  and  the  effects  of  tempera¬ 
ture  over  the  conditioned  range  of  -40  to  160*7.  Thus,  sach  source  was 
tested  and  its  effects  observad.  It  was  further  presumed  that  these 
Independently  derived  effects  could  bs  summed  to  express  the  net  affect. 
This  approach  is  illustrated  in  the  slide  by  the  multiple  calibration 
device#. 

III.  REVISED  CALIBRATION  METHOD.  Time  will  not  permit  a  detailing 
of  the  extensive  calibration  program  by  which  it  was  determined  that  this 
latter  premia*  was  invalid.  Perhaps  it  is  sufficient  to  say  that  this 
led  to  a  phase  that  is  often  referred  to  as  an  agonizing  reappraisal.  If 
the  basic  premise  that  there  were  no  synergistic  characteristics  was  in¬ 
valid,  how  then  could  the  stand  be  calibrated.  Alternate  schemes  were 
considered  but  the  one  adopted  involved  the  use  of  an  existing  program 
from  the  Computation  Center  files.  This  program  -  a  so  called  Nonslmple 
Stepwise  Multiple  Linear  Regression  Analysis  shortened  herein  to  MLR 
Analysis  was  used  to  fit  the  data  to  empirical  expressions  of  the  input 
components  in  terms  of  the  outputs.  These  expressions  need  be  limited 
only  by  the  capacity  of  the  program  which  permits  up  to  59  independent 
terms.  The  scheme  was  attempted  on  data  then  available  to  us  and  a  vary 
close  fit  was  obtained.  This  data  did  not  cover  the  full  ranges  of 
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interest.  hnuev«r  «n  a  raro l!br2ticn_plcn  vcc  formulated  end  leading 
hardware  was  designed  to  permit  the  acquisition  of  data  sufficient  for 
this  purpose.  The  plan  was  first  implemented  in  January  1969  for  ambient 
temperature  condition  and  analysis  of  results  were  completed  in  March 
1969.  Due  to  a  stand  renovation,  a  repeat  of  ambient  condition  cali¬ 
brations  have  been  made  as  well  as  a  calibration  at  each  of  the  temperature 
.extremes.  Analysis  is  in  progress  on  this  data.  The  scheme  developed  for 
the  January  1969  effort  was  used  thru  out  and  is  the  primary  subject  to 
be  presented  to  this  panel.  * 

The  loading  scheme  for  this  series  of  tests  involved  a  series  of 
input  vectors  whose  locations,  attitudes  and  magnitudes  were  closely 
controlled  and/or  measured.  The  input  assembly,  (Figure  3)  starting  at 
the  "hard"  point,  involved  a  bi-directional  translation  device  with 
planned  displacements  Indexed  by  a  series  of  dowel  pin  holes.  This 
device  was  centered  on  the  vertical  centerline  through  the  use  of 
optical  tooling.  Upon  this  device  was  mounted  a  hydraulic  jack. which 
was  linked  to  the  input  load  cell  through  a  universal  flexure.  A  rod 
extended  from  the  load  cell'  to  another  flexure  near  the  engine  mounting 
fixture  and  another  smaller  bi-directional  translation  device  connected 
this  flexure  to  the  mounting  fixture.  It,  too,  was  indexed  for  the 
planned  displacements.  During  loading  operations,  initial  displacements 
were  set  with  these  translation  devices  to  affect  either  vector  displace¬ 
ment,'  vector  angularity,  or  a  combination  of  both.  Then,  to  assure  that 
any  change  in  these  initial  conditions  was  known,  displacement  gages  were 
used  to  monitor  any  lateral  displacements  above  the  upper  flexure  or  below 
the  lower  flexure.  .  The  loads  were  then  cycled  under  control  of  a  servo 
loop  to  create  the  load  sequence  depicted  in  Figure  4.  To  combine  the 
static  pressure  effects  each  of  the  four  cycles  were  run  at  differerit 
line  pressure  conditions  as  noted.  Four  cycles  of 'this  type  constituted 
a  teat  run.  The  recalibration  plan  involved  30  runs. 

Digital  data  acquisition  and  processing  techniques  were  employed 
throughout  the  calibration  process  except  for  the  displacement  gages 
used  to  monitor  the  input  rod  attitude.  The  analog  data  obtained  from 
these  were  manually  reduced  and  entered  into  the  digital  analyses.  Many 
of  the  features  contained  within  the  computer  programs  are  illustrated 
in  Figure  5.  Automatic  normalizing  of  all  output  data  at  the  zero  input 
point  eliminates  the  negative  thrust  portions  of  our  load  cycles.  Redundant 
data  sectiona  are  also  eliminated  by  an  edit  routine.  "  The  low  range  bridge 
of  the  input  cell  defines  the  sustain  level  sections  indicated  while  the  high 
range  bridge  defines  the  original  boost  level  segments  indicated  by  solid  black 
lines.  A  range  limiting  feature  that  was  subsequently  added  redefines  the 
boost  level  to  conform  to  the  designated  upper  range  segments. 

For  each  of  these  segments,  the  200  sample/aec  acquisition  rate  was 
reduced  by  editing  and  averaging  to  approximately  1  sample/sec  for  boost 
data  and  10  a/s  for  sustain  data.  Computation  of  input  and  output 
components  were  then  made  at  these  data  rates  and  taped  for  input  into 
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Che  MLR  Analysis.  Inputs  were  based  on  initial  and  monitored  change 
of  rod  attitudes  while  outputs  were  functions  of  the  stand  geometry 
and  the  load  cell  results,  tabulations  and  CRT  type  plots  were  pre¬ 
pared  to  aid  in  analysis. 

The“  MLR  Analysis  determined  the  empirical  equation  coefficients 
for  each  component  in  turn.  Data  from  25  of  the  30  calibration  runs 
was  considered  collectively.  The  determined  coefficients  were  then 
programmed  to  provide  predictions  of  the  input  components  which  were 
then  compared  to  the  known  inputa?and  a  residual  error  computed.  When 
applied  to  runs  26  thru  30,  a  validity  check  was  made  of  the  total 
process.  When  applied  to  all  30  runs  in  turn  a  standard  error  is 
obtained  for  each  run.  It  is  the  combination  of  these  errors  that 
determined  our  estimate  of  measurement  system  capability. 

Several  forms  of  the  MLR 'Analysis  were  performed  as  indicated  in 

Table  II.  To  understand  this  table,  you  must  realize  that  these  are 

the  stand  output  parameters  that  .are  considered  as  independent  vari- 

ablee  in  each  of  six  equations  Involving  the  known  input  values  or 

dependent  variables.  The  equation  form,  as  noted,  la  Input  Component  ■ 

I  AT  with  A  the  unknown  coefficients  and  T  the  terms  selected  from 
n  n  n  n 

the  chart.  The  analysis  is  performed  on  each  of  these  equations  in 

turn  to  determine  a  best  fit  set  of  unknown  coefficients,  one  for  each 

of  the  independent  terms.  Having  determined  these  values  by  calibration, 

it  is  assumed  that  the  equation  Holds  for  unknown  input  conditions  and 

thus  these  conditions  can  be  predicted  from  any  set'  of  output  values 

contained  within  the  calibration  ranges.  The  original  premise  was  that 

Irrelevant  terma  would  be  effectively  eliminated  by  the  analysis  and  ,a 

59  term  form  was  chosen^  The  highest  order  terms  used  were  second  degree 

terms  and  their  crose  products.  This  provided  a  very  good  fit  for  the 

25  runs  used  in  the  analysis  but  prediction  for  "runs  26  thru  30  were 

very  poor,  it  was  reasoned  that  this  was  due  to  the,  inclusion  of  too 

many  Irrelevant  terms  and  the  equations  were  reduced  to  a  16  term  form. 

(Diagonal  shading1!)  All  cross  products  are  eliminated.  '  This  produced' 

significantly  Improved  prediction  results  even  though  the  standard 

errors  from  the  MLR  Analysis  were  increased.  ’This  form  was  adopted 

for  the  sustain  range  of  data.  An  addendum  notp  should  be  made  at 

this  point.  In  setting  the  program  controls,  no  test  was  made  on  the 

exclusion  of  terms.  Present  analysis  includes  this  feature  but  ita 

effectiveness  is  not  yet  known. 

The  next  significant  step  was  taken  when  it  was  observed  that  even 
though  the  standard  error  for  an  over-all  fit  was  acceptable,  the  fit 
at  the  upper  limit  of  the  range  was  at  times  unacceptable.  This  led  to 
a  revised  analysis  baaed  on  data  contained  within  the  Interval  of  25  to 
&0K.  The  16  term  form  was  selected  for  this  analysis.  The  residual 
error  plots  seemed  to  suggest  a  correlation  with  the  ratios  of  Z  Input 
with  each  of  the  components.  Thus,  two  forms  containing  23  terma  in  the 
equations  were  attempted  next.  These  forms  include  the  16  termB 


;in'J  thjse  shown  In  dots  In  Table  II.  Alternates  9hown  involve  the 
products  or  ratios  with  "Z".  Results  were  only  slightly  improved  in 

. c t >u£  iji  lIic  at.  uiuuiu  t:iT Luio  OuLttiucu,  ho wever  thr  reel duSl  CTTCT 

plots  pointed-up  a  pretevence  for  either  ot  these  over  the  16  term 
torn  for  most  if  the  alx  components.  This  will  be  demonstrated  in 
tutuie  figures.  Attempting  further  Improvement,  two  additional  forms 
wlln  33  teems  vj.'Te  used,  lhe  first  altenpi  uldeJ  L!i«*  torsis  identified 
with  horizontal  shading.  The  second  was  broken  into  two  sub-analyses  - 
the  16  terra  for:n  plus  an  analysis  of  the  residual  errors  in  terms  of 
the  balance  of  17  terms.  Significantly  improved  standard  errors  were 
obtained  with  the  first  of  these  but  again  the  results  of  both  produced 
poor  predictions  for  the  confirmation  runs.  A  comparative  charting  of 
the  five  forms  /is  tney  upp  ■.  1  to  run..  26  t.irvugh  30  are  shown  in  Table  III. 
The  bear  equation  results  are  in  shaded  areas  for  each  component.  The  16 
term  forr.  fox  Wind  MX;  t.he  .V!  term  for  MX  aid  the.2'--  :tre  ratios  t'or  the 
Pi  1  iii'i  cu .  Tr'e  ifipi;1  for  Run  27  which  dte  typical  ol  the  runs  examined 
will  il  -Htr.i*.*  I'.rt  the  reasoning  for  cur  prefetences.  Tries«  ate 
residual’  error  ?  L  input  nlots  from  the  16  term  analysis  with  flat;, 
points  plotted;  A  computer  determined  best  fit  is  line;  plotted  thru 
these  points  and  appears  as  a  broken. line.  Similarly jietermined  beet 
fU  plots  for  each  of  the  23  terra  forma  have  been  manually  transferred 
to  these  plots  far  comparison. .  Comparison  should  be  made  between  the 
line  plots.  The  ideal  result  would  appear  as  a  zero  error  through  out 
the  range,  Figure  6  is  the  X  component  plot.  The  23  term  results  are 
obviously  best  with  maximum  errpr  for  the  mean  data  fit  of  approximately 
3  pounds.  Figure,  7  is  the  Y  plot  with’  a /  maximum  error  of  8  pounds.  The 
7.,plots  (Figure  8)  show  a  slight  preference  for  the  23  term  ratio  form,  , 

•  All  data  is  under  50.  pounds  of  error  in, the  presence  of  Inputs  ranging 
to  4QK  of  thrust,  700  lbs.  of  side  foreje  end  2000  ft-lbs  of  moments.  The 
MX  plot*  (Figure  9)  show  a  mean  error  of  lees  than  30  ft- lbs.  The  MY 
plots  (Figure.  10)  show  error  ranging  to  100  ft-lbs  for  this  run  but  this 
is.j somewhat  1‘arger  than  the  normal  MY  error.  The  MZ  plots  (Figure  11) 
show  little. preference  between  these  .three  forms  but  the  standard  error 
is  leqs  than'  4  ft-lba  for  ell  forma^  1  * 

The  foregoing  description  suggests  a  much  more  direct  path  than  that 
of  our  actual  experience.  From  an  experiment  standpoint  there  were 
several  areas  that  could  have  been  explored  in  greater  depth  but  were 
ignetfred  since  this  was .primarily  an  attempt  to  define  the  specific 
capabilities  of  the  stand  as  they  related  to  the  LANCE  facility  re¬ 
quirements.  A  final  summary  of  the  attainment  of  this  goal  is  presented 
in  a  comparison  of  the  facility  requirement*  with  the  demonstrated; 
capability  (Table  IV).  Based  on  these  results  and  our  continued  ;  . 

surveillance  of  the  stand  through  verification  testing,  we  are  con¬ 
fident  that  our  data  quality  must  be  rated  equal  to  or  superior  to  that 
obtainable  at  any  facility  of  this  type. 

A  concise  summary  of  the  major  points  of  the  calibration  problem,  the 
implemented  solution  and  the  clinical  questions  to  be  asked  of  this  panel 
is  now  In  order. 
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THE  PROBLEM,  Since  It  la  deal rad  to  reaolve  a  fore*  vactor  in 
aix  dagraaa  of  fraadoa  It  la  neceeeary  Co  avaluata  tha  interaction 

ui  lira  •lactroMchanicai  Maauraaant  ay  at  as  in  aix  dagraaa  at 
freedom.  For  tnla  purpoaa  an  axpariMnt  la  required  that  will  adequately 
meaaure  theae  affacta  throughout  tha  ranga  of  lntaraat  and  determine  tha 
pracialon  charactarlatlca  of  tha  total  Maauraaant  procaaa. 

THE  IMPLEMENTED  SOLUTION.  ; 

A.  An  axpariMnt  of  30  Input  vactora  Involving  dlffarlng  combination* 

of  tha  aix  component*  of  forea  input  and  atatlc  praaeurlsatlon 
lavala  with  ranging  of  aach  paraMtar  to  naar  maximum  axpactad 
valua.  4 

B.  An  evaluation  of  the  aix  empirical  “boat  fit"  tranafar  aquatlona 
relating  tire  observed  input  and  output  data.  Tha  taraa  of  tha 
aquatlona  w*rj  arbitrarily  aalactad  and  a  goaf f lent*  derivation 
made  by  a  multiple  linear  ragraaalon  analyaia. 

C.  A  derivation  of  pracialon  llalta  baaed  on  tha  combination  of 
tha  overall  axpariMnt  data  fit  pracialon  with  atahdard  devia¬ 
tion  valuaa  for  the  laboratory  acandard  and  for  tha  tranafar  to 
tha  flaid  atandard.  Thla  coaibnatlon  la  by  tha  aquara  root  of 
tha  aua  of  varlancaa  Mthod. 

THE  CLINICAL  QUESTIONS. 

V 

1.  la  method  valid? 

2.  If  ao,  are  tha  thirty  taata  axcaaalva  or  inadaquata? 

«  . 

3.  Are  more  practical  mathode  known?  j 

■'  / 
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TABLE  I.  REQUIREMENT  VERSUS  CAPABILITY  ESTIMATE 
-  AT  START  OF  XRL  PROGRAM 


} 

Parameter  $ 

Requirement 
(!)  Figma 

Capability 

Estimate 

(!)5%  confidence* ) 

Side  forces  (lb) 

Sustain  phase  thrust  (lb) 

Boost  phase  .thrust  (lb) 

Boost  phase  vector  location  (in. ) 

Boost  phase  vector  angularity  (m  rad ) 

±  50 

±  160 
±?00 
±0.03  , 
±0.372 

±  loss’ 

t  100 

*  250 
±0.  125 
±2.6 

I 

I 
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PROPELLANT 
ft£D SYSTEM 
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SYS 


BKHRECTION 

TRANSLATOR 


HVjRAULIC 

JACK 


UNIVERSAL 

FLEXURE 


INPUT  OUAL  RANGE 
LOAD  CELL 


UNIVERSAL 

FLEXURE 


OI-OIRECTION 

TRANSLATOR 


ENGINE 

MOUNTING 

FIXTURE 


CENTER  OF 
FLEXURE 


106.42  iir. 


*2 


CENTER  OF 
FLEXURE 


S.4S  in. 


CENTER  OF 
RESOLUTION 


FIGL'KE  3.  CALI HKATIOV  FORCE  INPUT  ASSEMBLY 
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SEGMENTS  OF  LOADING  SEQUENCE  CONSIDERED  R^  ANALYSIS  PROGRAM 


INPUT  Z 


rii'.i  m; 


HE.  M.  KUIUK  Pl.OTS,  X  COMPONENT 


EQUATICN  I  X  COMPONENT  (lfr£  |  T  COMPONENT  Db'  I  Z  COMPONENT  ^ 


CORRECT  EO  DELTA 


CORRECTED  DELTA  Z 


23-TERM  PRODUCTS 
23-TERM  RATIO 
16-TERM  SOLUTION 


hwi 

■ram 


iraBraMramravrau 
^■i■■■ra■fi■ra^a■raSil 

■MMHnMmmKttraranM 

■iiiHnnimamnnwi 

wBraranraraiinraranm 


BniicanninMiiv^ini 
nw.ummmwmmnmmwm<ammmmwmm 

■mSlMHMIHPPCaMHHMHI 
■■P9fllimBlUPr:2fl|;||MiaHM 

■C«HraX*7^raMMnraHnm 

i*M»=:=;«*>£2mnHHHMHHH 

kaillDlllllllllliuilillllll 


2,000  -30,000  -28,000  -26,000 

PUT  Z 

PLOTS,  MY  COMPONENT 
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rABLfc  IV.  REQUIREMENT  VERSUS  DEMONSTRATED  CAPABILITY 
AT  END  OF  CALIBRATION  ANALYSIS 


Su  staii 


Parameter 


Side  force  fib) 

Vertical  i-'.rcc  (thrust)  (IK) 
V  ?et or  location  (in.  i 
v-  ctor  angularity  enrad) 


>;de  force  (1b) 

Vertical  force  (lb) 

Vector  location  (in. ) 
Vector  angularity  (mrad) 


Requirement  Capability 


t  SO 
-  200 
-  0.03 
r  0.  372 


+  13 
IK) 

•  o.  or, 

•  1.2 


I ME  3  -.MORATORY  STUDY  OF  A  METHOD  F  )H  Ml  A1.  UR  INC 
AMMONIUM  PERCIT-JRAfE  PART  I  Cl.F  SIZE 


Bernard  J.  Alley 

Army  Propulsion  Laboratory  and  Center 
Research  and  Engineering  Directorate 
U.  S,  Army  Missile  Comnand 
Redstone  Arsenal,  Alabama  35809 


ABSTRACT 


An  in  ter  labor  a  ur,  study  of  a  method  ior  measuring  the  particle 
size  distribution  of  finely  ground  ammonium  perchlorate  was  conducted 
by  the  Analytical  Chemistry  Working  Group  of  the  Interagency  Chemical 
Rocket  Propulsion  Group  (ICRPG).  The  primary  objective  of  the  study 
was  to  determine  the  suitability  of  the  method  for  use  as  a  standard 
specification  procedure  by  evaluating  its  precision.  Single  analyses 
of  two  different  ammonium  perchlorate  samples,  having  weight  median 
diameters  In  the  range  of  20-30u  were  made  by  each  of  nine  laboratories, 
using  the  same  liquid  sedimentation  technique  and  equipment.  The 
random  error  within  laboratories  and  the  systematic  error  among 
laboratories  were  resolved,  and  confidence  intervals  were  placed  on 
the  determination  of  specific  surface  areas  and  weight  mean  diameters. 
The  random  error  estimate  was  acceptably  small;  however,  the  system¬ 
atic  error  estimate  was  so  large  that  the  method  is  not  recommended 
for  use  as  a  standard  specification  procedure. 


INTRODUCTION 

Ammonium  perchlorate  (AP)  Is  widely  used  as  an  oxidizer  In  com¬ 
posite  and  composite-modified  double-base  propellants.  The  particle 
size  distribution  of  the  AP  has  a  pronounced  effect  on  the  propellant 
processing  characteristics  and  ballistic  properties,  and  therefore 
must  be  precisely  measured  and  controlled.  The  recent  use  of  finely 
ground  AT  In  high  burning  rate  propellants  places  greater  demands  on 
the  precision  of  particle  size  analysis. 


This  article  has  been  reproduced  photographically  from  the  author's 
manuscript. 
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Preceding  pege  blank 


■vrrjl  .irvily>:s  methods  were  evaluated  and  compared  during  a 
!•"  ’<?:  •'  i  the  Arry  Prnpul  sin-i  Laboratory  and  Center.1 

1  >-••  •'  ■•  ;  ’  ■  ,  K  on  a  liquid  sed invitation  analysis  using 

an  npr>  u.:  r -t  n  l  1  ■  •  rurod  by  chc  Mine  Safety  Appliances  (M-S-A) 

Conparr  ,  gave  precise  results  and  was  recommended  for  the  general 
analysts  of  finr  A?.*  The  method  was  used  with  success  by  a  number 
of  laboratories  throughout  the  propulsion  industry. 

The  interlaboratory  study  (Round  Robin)  described  here  was 
subseouently  conducted  by  the  Analytical  Chemistry  Working  Group  of 
the  Interagency  Chemical  Rocket  Propulsion  Group  (ICRPG) ,  with  nine 
laboratories  participating.  Ine  objectives  were:  (1)  to  determine 
the  suitability  of  tie  M-S-A  method  for  use  as  a  standard  specifica¬ 
tion  proc.d  to  h-ised  in  nr.  escimarior  of  its  precision;  (2)  to  deter- 
r .  1 *  '.i  effeo-lvcnes  of  i-  simple  experimental  design;  and  (3)  to 
k'.ilu.’.Cc  a;vt  tcr.p.iVv  »:  performances  of  the  partic ipatirg  laboratories. 


EXPERIMENTAL 

Each  of  the  nine  laboratories  was  sent  three  sanqplee  of  nominal 
20- 30u  AP  mixed  with  an  Inert  polymer,  one  selected  at  random  from 
each  of  three  different  batches.  One  of  the  samples  was  provided 
a  imp  ly  for  practice  prior  to  initiation  of  the  Round  Robin.  The 
other  rwo,  designated  materials  A  and  B,  were  to  be  analyzed  in 
accordance  with  tne  detailed  Round  Robin  procedure.  The  instructions 
spue  If  Led  that  the  analysis  be  conducted  by  a  skilled  operator,  and 
thut  the  A  and  R  samples  be  analyzed  on  differeAt  days. 

briefly,  the  particle  analysis  procedure1  was  as  follows, 

A  l 5-rrg  sample  of  the  material  was  dispersed  with  a  surfactant  and 
suspended  in  a  feeding  liquid  composed  of  60%  chlorobenzene  and 
A 07,  benzene  by  volume.  The  particle  suspension  was  placed  on  top 
of  chlorobenzene  In  a  special  centrifuge  tube.  The  larger  particles 
were  allowed  to  fall  under  the  influence  of  gravity,  end  the  smaller 
particles  were,  centrifuged.  All  of  the  particles  were  collected  in 
■  uniform  bore  capillary  at  the  bottom  of  the  centrifuge  tube.  The 
diameter  schedule  used  in  the  Round  Robin  and  a  typical  analysis  are 
shown  in  Table  1. 

The  sediment  height  a',  each  particle  diameter  of  the  schedule 
was  .leasjru!  as  a  function  ot  settling  times  precalculated  from 
StoV.es  law.  The  percentage  by  weight  (volume)  of  particles  greater 
than  each  successive  diameter  was  calculated  by  dividing  the  corre¬ 
sponding  sediment  height  by  the  total  height  at  the  end  of  the 
analysis.  It  will  be  noted  from  the  table  that  the  percentages  are 
not  Independent. 
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Table  1 


T' 

-T  i  ; 
wi  th 

1  Ammonium  Perc’ilorate  Parti 
the  Mine  Safety  Appliances 

clo  Siz«.-  Analysis 
(M-S-A)  Analyzer 

Diameter 

M 

Sed imentation 

Sedimentation 

Time 

(min,  sec) 

Sediment 

Height 

(Relative) 

Weight  7.  > 
Diameter 

w  ,  Rate 

Mode  ,  , 

(rpm) 

200 

Grav 

ity 

0,  4.1 

0 

0 

149 

0,  7.5 

0.5 

1.45 

103 

0,  15.0 

1.0 

2.90 

74 

0,  30.2 

2.2 

6.2S 

52 

1,  1 

5.5 

15.94 

37 

2,  1 

10.8 

31.30 

25 

> 

t 

4,  25 

16.5 

47.83 

18 

Cent 

:rifuge  300 

0,  27 

21.0 

60.87 

9 

600 

0,  58 

26.2 

75.94 

5 

1200 

9,  52.5 

29.5 

85.51 

3 

1800 

l.  19 

31.5 

91.30 

2 

2,  2 

32.5 

94.20 

1.2 

5,  16 

33.5 

97.10 

0.6 

3600 

6,  2 

34,2 

99.13 

0.4 

* 

f 

9,  55 

34.5 

100 

RESULTS  AND  DISCUSSION 


The  particle  aize  distribution  data  are  given  in  Tables  II  and  III. 
The  data  for  laboratories  3  and  8  were  omitted  from  the  calculations 
of  averages  and  the  estimates  of  variances  (Ss)  and  standard  deviations 
(S)  because  of  the  outliers4  in  Table  111  and  the  abnormal  shapes  of 
their  particle  sice  distribution  curves. 

The  average  particle  size  distribution  data  are  plotted  on  log- 
probability  scale  in  Fig.  1.  The  shapes  of  the  curves  are  typical  of 
those  obtained  for  finely  ground  unimodal  ammonium  perchlorate.  The 
difference  between  tne  particle  size  distributions  of  the  AP  in  the 
two  materials  was  purposely  made  small  so  that  the  random  errors  for 
their  respective  analyses  could  be  assumed  to  be  equal. 
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Table  11 
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Fig.  1.  Ammonium  Perchlorate  Particle  Size 
Distribution  Curves 


Of  the  large  number  of  single- valued  variables  that  can  be 
calculated  from  the  particle  sice  distribution  data,  the  two  chosen 
for  this  program  were  specific  surface  area  (Sy)  and  weight  mean 
diameter  (dy) .  The  specific  surface  area  correlates  well  with 
propellant  burning  rates1  and  is  very  sensitive  to  variations  in  the 
diameters  of  small  particles;  the  weight  mean  diameter  is  very  sensi¬ 
tive  to  variations  in  the  diameters  of  large  particles.  The  Sy  and 
dy  values,  assuming  spherical  particles,  were  calculated  from  the 
data  in  Tablet  II  and  III  by  the  following  formulas: 


S^n^/g) 


3.077 


n  Wj 


dw^> 


l  di  . 


(1) 

(2) 


whore  w,  la  the  weight  fraction  of  particles  in  cHe  1  else 
Interval',  and  is  an  average  diameter  of  the  1th  interval. 
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procedures  developed  by  Youden.6 ,e  A  two-material  XY  plot  tor  ^  is 
shown  in  Fig.  2.  Horizontal  and  vertical  lines  were  drawn  through  the 
medians  of  the  points,  and  a  45°  line  was  drawn  through  their  inter¬ 
section.  The  perpendicular  distances  between  the  points  and  the 
45°  line  are  a  measure  of  the  random  error  within  laboratories,  and 
the  spread  of  points  along  the  45°  line  is  a  measure  of  the  systematic 
error  among  laboratories. 
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Fig.  2.  Two-Material  Plot  of  Ammonium  Perchlorate 
Specific  Surface  Areas 


•CM. 


The  arrangement  of  points  shows  that  the  laboratories  tended  to  get 
either  high  or  low  results  on  both  materials.  Moreover,  the  systematic 
error  Is  appreciably  larger  than  the  random  error.  Most  of  the  labora¬ 
tories  have  a  small  random  error,  indicating  that  they  did  careful  work. 
The  differences  between  the  two  material*  for  laboratories  3  and  5  were 
found  to  be  statistically  significant  at  th*  957.  confidence  level  when 
conpared  with  the  average  difference  for  ell  laboratories. 
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riK'jro  3  is  cho  two-material  XY  plot  for  dy.  The  laboratory  8 
vmuf  differ;  narked ly  from  the  others,  particularly  for  the  analyala 
of  cute:, ial  b,  and  was  not  considered  when  drawing  the  horizontal  and 
vertical  lines.  The  random  error  for  dy  appears  to  be  greater  than 
that  for  Sy.  It  should  be  recognized,  however,  that  the  dy  values 
are  larger  and  that  dy  and  Sy  were  calculated  from  the  seme 
particle  size  distribution  data.  The  systematic  error  among  labora¬ 
tories  is  not  noticeably  larger  than  the  random  error  within 
laboratories. 


45  r 


Fig.  3.  Two-Material  Plot  of  Ammonium  Perchlorate 
Weight  Mean  Diameters 


The  extremely  high  dv  value  for  laboratory  8  clearly  indicates 
some  deviation  from  the  recommended  procedure,  and  this  laboratory 
was  omitted  from  some  of  the  statistical  analyses  described  later. 
Laboratory  3  was  also  omitted  from  some  of  these  analyses  because  of 
its  low  dw  value  and  the  outliers  and  abnormally  shaped  curve* 
mentioned  earlier. 
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6  7 

An  analysis--  of  variance  ’  la  given  in  Table  IV.  The  results,  as 
expected,  agr^e  with  the  qualitative  interpretations  ot  the  data  fr. 
iigs.  2  and  3.  The  variance  of  Sy  averages  among  laboratories  is 
statistically  significant  when  compared  with  the  estimated  random  error 
variance  (S^);  whereas  that  for  dw  is  not  statistically  significant. 
The  laboratories  x  materials  mean  square  (MS)lm  is  considered  to  be 
the  best  estimate  of  the  true  random  error  variance. 


Table  IV 

Analysis  of  Variance  (Laboratories  3  and  8  Omitted) 


DF  -  degrees  of  freedom  MS  -  mean  square 

SS  -  sum  of  squares  EMS  -  expected  mean  square 

SR  *  Se  +  “  (HS)^ 

c3  (MS)l  -  Sr 
bL  “  2 

Sd  “  SL  +  4/” 

Statistically  significant  at  the  957.  or  higher  confidence  level. 
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The  estimate  of  the  systematic  error  variance  among  laboratories 
(S£)  is  also  an  Important  component  of  the  estimate  of  the  overall 
variance  (Sj)  of  the  method.  The  formulas  for  calculating  S*  and 
are  given  In  the  table.  The  expected  mean  squares  (EMS)  are  for 
a  random  model,  vhich  was  assumed  in  this  case. 

Estimated  random  and  systematic  error  variances  ara  given  In 
Table  ^  for  various  confcinations  of  ths  Sy  and  dy  data.  The  sS 
and  values  for  specific  surface  area  determinations  ara  not 
significantly  affected  by  omitting  laboratories  3  and  8,  but  they  ara 
significantly  reduced  In  the  ceae  of  weight  wan  diameter  determinations. 
The  value  of  for  the  determination  of  specific  surface  area  ia 
significantly  reduced  when  laboratories  3  and  5  ara  omitted,  aa  could 
have  been  expected  /torn  Fig.  2.  However,  a  comparison  with  the  0.00137 
value  Independently  tb’alned  by  replicate  determinations  within  labora¬ 
tory  1  (the  originating  laboratory)  Indicates  chat  the  value  of 
0.000048  ia  not  a  good  estimate  of  the  true  random  error  variance. 


Table  V 

Estimated  Random  and  Systamtlc  Error  Variance* 


Sr 

Sl 

Specific  Surface  Area  (^/g) 

All  Laboratories  { 

0.001171 

0.006192 

Omit  Laboratories  3  and  8 

0.000779** 

0.006328b 

Omit  Laboratories  3  and  S 

0.000048 

m  m 

Within  Laboratory  1* 

0.001374 

— 

Weight  Mean  Di 

amatar  (j*> 

All  Laboratories 

48.78 

62.94 

Omit  Laborstorles  3  and  8 

7.60*> 

10.25** 

Within  Laboratory  la 

1.73 

*  m 

*DF  -  5. 

“Values  used  for  calculating  confidence 

intervals. 

/ 


a 

The  Sg  fsT  She  rSpliCiU  wMliioauua  of  weight  mean  diameter 
within  laboratory  1  la  significantly  saallar  at  the  957.  confidence 
level  than  s£  for  tha  Round  Robin  data  with  laboratories  3  and  8 
omitted. 


The  precision  of  the  method  is  defined  by  the  confidence  limits 
(X  ±  L^),  where  X  la  an  analysis  result  and  L^  is  one-half  the 
lzr>gch‘of  the  confidence  interval.  In  this  case  at  the  95%  level. 

These  one-half  values  are  given  In  Table  VX.  They  were  calculated 
from  the  formulas  in  tha  table  and  the  estimated  s|  and  SL  variance 
components  noted  in  Table  V.  The  degrees  of  freedom  associated  with 
Sd  were  estimated  by  Satterthwaltea  approximation.7 


Table  VI 

Precision  of  M-S-A  Analysis  Method 
(L^  at  the  95%  confidence  level) 


Number 

of 

Analyses 

(n) 

Specific  Surface  Area  (ma/g) 

Weight  Mean  Diameter  (|i) 

Within  a 
Laboratory 

Laboratory 
at  Random  0 

Within  a 
Laboratory 

Laboratory 
at  Random  0 

1 

0.0683 

0.201 

6.75 

'  9.55 

2 

0.0483 

0.196 

4.77 

8.47 

3 

0.0394 

0.194 

3.89 

8.08 

4 

0.0341 

0.193 

3.37 

7.88 

tf  Sg  ,  where 

,  with 

tf  la  Student's  t 
f  degrees  of  freedom. 

(f  -  6) 

%  “ 

tf  sd 

(f  -  6.69  for  specific  surface  area, 
f  ■  9.02  for  weight  mean  diameter). 

The  estimated  precision  of  analysing  AP  saag>les  within  a  single 
laboratory  (randosi  error)  la  given  by  the  confidence  intervals  in  the 
second  and  fourth  columns  of  Table  VI.  These  Intervals  apply  for  the 
analysis  of  AP  series  having  particle  else  distributions  within  the 
range  shown  In  Fig.  1.  Note  that  the  precision  lq?roves  with  an 
increase  In  the  number  of  replicate  analyses. 
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The  estimated  precision  of  analyses,  considering  the  random  plus 
tne  systematic  error,  by  any  laboratory  selected  at  random  la  given  by 
Che  confidence  intervals  In  the  third  and  fifth  columns.  Assuming  the 
participating  laboratories  are  representative  of  the  entire  population 
of  laboratories,  these  precision  estimates  determine  the  suitability 
of  the  method  for  uee  es  e  standard  epaeification  procedure.  The 
error  is  larger  than  for  analyses  within  a  single  laboratory  because 
of  the  contribution  of  the  systematic  error  variance  (S^).  Nor  la  the 
precision  improved  much  in  thie  case  by  replicate  analyses,  because 
the  replicates  (n)  reduce  only. the  smaller  Sg  component  of  Sj. 

Perhaps  in  actual  practice  a  higher  degree  of  confidence  than  95% 
would  be  desired.  lor  higher  degree#  of  confidence  the  value  of  tf 
would  be  larger,  and  the  confidence  intervals  would  increase  accord- 
ingly.  The  accuracy  (bias)  of  the  method  could  not  be  estimated  in 
this  Round  Robin,  bec.uae  &  standard  AP  sample  of  accurately  known 
particle  size  is  not  available. 


WEIGHT  PERCENT  GREATER  THAN  DIAMETER 


Fig.  4.  95%  Confidence  Intervals  on  the  Percentage 

Points  of  the  Particle  Sice  Analysis  of  Aamonium 
Perchlorate  in  Materiel  A  by  Laboratory  1 
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cf  a  ; article  size  distribution  curve.  This  .s  il luc trn ted  in  Fig.  4 
ior  t>'-  analyst:.  of  a  single  sample  of  material  A  by  1  at  oratory  1. 

"he  confidence  intervals  were  calculated  using  the  variance  estimate* 
in  labia  II.  Such  a  precision  estimate  is  of  \aiue  for  determining 
wnether  the  variations  in  particle  size  distribution  .ire  due  to  the 
analysis  procedure  or  the  grinding  process. 


CONCLUSIONS  AND  RECONHENDATIONS 


The  precision  of  analyses  within  a  single  laboratory  is  considered 
adenuate .  and  the  f act  that  a  number  of  laboratories  are  successfully 
using  the  procedure  supports  this  conclusion  However)  the  method  i.> 
not  recommended  os  a  standard  specification  procedure  rot  the  pa"t  cle 
size  analysis  of  :imn  .nal  20-30u  atnnonlum  perchlorate,  because  c*  ,;i- 
large  systematic  arror  among  laboratories  in  the  determination  of 
specific  surface  area.  The  great  difference  between  these  two  errors 
could  be  due  to  some  deficiency  in  the  analytical  procedure  that  permits 
laboratories  to  introduce  their  own  variations.  Qne  likely  area  of 
inconsistency  is  in  the  dispersion  of  the  AP  particles,  but  there  are 
no  known  alternative  techniques  that  would  not  also  affect  the  accuracy 
'of  analyses. 

The  simple  experimental  design,  without  replication,  encouraged 
laboratories  to  participate,  thus  enabling  a  more  reliable  estimate 
of  systematic  error  among  laboratories.  Past  experience  has  shown  that 
this  systematic  error  Is  almost  always  significantly  greater  chan  the 
random  error  within  laboratories. 

The  comments  and  suggestions  of  the  clinical  session  panelists  are 
particularly  solicited  with  respect  to  the  following  elements  of  the 
Round  Robin  statistical  analysis: 

(1)  Estimation  of  the  degrees  of  freedom  associated  with 
S<p  and  Sj_ . 

(2)  Determination  of  confidence  intervals  oi  regions  tor 
particle  size  distribution  curves. 

(3)  Criteria  for  the  rejection  of  extreme  laboratories  and  data. 

(4)  Experimental  design  for  Round  Robins  and  possible  alternatives. 
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NEW  ANALYSES  AND  METHODS  LEADING  TO  IMPROVED  TARGET 
ACQUISITION  REQUIREMENTS  INVOLVING  SYSTEMS,  GEODETIC 
AND  RE-ENTRY  ERRORS,  AND  INCREASED  WEAPONS  EFFECTIVENESS 
FOR  CONVENTIONAL  WEAPONS* 

Hans  Baussus-von  Luetzow 
U.  S.  Army  Engineer  Topographic  Laboratories 
Fort  Belvolr,  Virginia 

SUMMARY.  After  a  cursory  critique  of  currently  used  methodology 
for  the  study ''f  target-accuracy  requirements  for  artillery  weapons, 
this  research  note  is  concerned  with  the  development  of  analytical 
methods  and  two  different  though  interrelatable  and  essentially  ad¬ 
ditive  optimization  concepts.  If  implemented  within  the  context  of 
TACFIRE,  these  methods  are  conservatively  estimated  to  provide  on  the 
average  a  30Z  greater  weapons  effectiveness.  Although  the  intra  and 
extra  weapons  systems  employment  parameters  are  Interdependent,  variable, 
and  changing,  an  integrated  operational  optimization  is  achieved.  The 
methods  outlined  are  also  useful  in  weapons  R&D  and  related  systems 
analyses.  Furthermore,  the  rather  cogent  requirement  and  related 
recommendations  or  conclusions  arrived  at  may  be  of  considerable 
significance  for  certain  R&D  and  combat  development  activities. 

FOREWORD.  It  was  originally  contemplated  to  finalize  this  study 
in  1967.  The  author  who  waa  also  Investigating  more  powerful  methods 
in  connection  with  burst  and  target  height  variabilities  and  the  use  of 
conventional  cratering  and  nucleat  weapons  became,  however,  increasingly 
convinced  that  rudimentary  or  short-cut  methods  had  to  be  considered  un¬ 
satisfactory.  A  more  rigorous  and  mature  approach  required  time  and 
concentration  in  view  of  the  slow  progress  made  in  the  past  and  also 
because  of  a  satellite  systems  study  performed  during  1967.  As  to  the 
word  "improved"  in  the  title,  this  should  rather  be  Interpreted  as  "less 
restrictive,"  Implementation  of  cite  methods  and  concepts  developed  would 
undoubtedly  lead  to  a  significant  increase  of  Army  weapons  effectiveness. 
In  addition,  the  new  methods  are  expected  to  have  some  ramifications  per¬ 
taining  to  a  variety  of  R&D  and  combat  development  activities.  The 
technical  responsibility  for  this  study  is  exclusively  the  author's  who 
appreciates  USAETL's  continued  interest  in  this  kind  of  effort. 


*This  article  appeared  as  Research  Note  No.  15,  U.  5.  Arm>  Engineer 
Topographic  Laboratories,  Fort  Belvoir,  Virginia. 

The  remainder  of  this  article  has  been  reproduced  photographically 
from  the  author's  manuscript. 
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NEW  ANALYSES  AND  METHODS  LEADING  TO 
IMPROVED  TARGET  ACQUISITION  REQUIREMENTS 
INVOLVING  SYSTEMS,  GEODETIC  AND  RE-ENTRY 
ERRORS,  AND  INCREASED  WEAPONS  EFFECTIVENESS 
FOR  CONVENTIONAL  WEAPONS 


I .  Introduction. 

1.1  The  essential  ideas  underlying  this  report  were  developed  in  August  1966 
after  an  evaluation  of  the  following  material:  “Target  Acquisition  Accuracy  Require¬ 
ments,  1965-1975  (U)"  (1)*:  “A  Model  for  Determining  Target  Location  Accuracy 
Requirements"  (2);  “Trip  Report  to  CDC  Artillery  Agency"  (3);  and  “A  Technical 
Analysis  to  Support  Map  Accuracy  Requirements"  (4). 

1.2  According  to  Ref.  (3).  additional  contractual  work,  to  start  in  July  1967  and 
expected  to  last  one  year,  was  considered  necessary  by  USAC.DC  in  order  to  improve 
the  methodology  report  (1).  It  led  to  the  report  “A  Study  of  Target  Location  Accur¬ 
acy  Requirements  for  Artillery  Weapons  -  Army  1975  (U)”  (5).  This  Btudy,  conduct¬ 
ed  at  the  Combined  Arms  Research  Office,  Fort  Leavenworth,  Kansas,  and  coordinated 
with  the  USACiM)  Artillery  Agency,  Fort  Sill,  Oklahoma,  applied  the  methodology  of 
Ref.  (1)  to  all  artillery  weapons  of  the  1966-1975  time  frame. 

1.3  The  methodology  in  both  Ref.  (1)  and  Ref,  (2)  is  essentially  restricted  to  the 
2-dimcnsionul  problem  of  fragmentation  projectiles  with  impact  fur.es  and  thus  less  ap¬ 
plicable  with  respect  to  height  bursts.  It  consists  of  computings  measure  of  effective¬ 
ness  f  (see  Ref,  (1),  11-1,  eq.  (2))  and  a  fractional  coverage  C  (Cl  in  Ref.  (2))  so  that  the 
fraction  of  casualties  F  =  f  •  C.  Although  it  has  not  been  spelled  out  explicitly,  f  is  the 
probability  of  hitting  the  target  which  is  computed  by  dividing  the  common  area  be¬ 
tween  target  and  effects  pattern,  a,  by  the  target  urea  Aj.  The  determination  of  f  in¬ 
volves  the  use  of  u  quantity  A|  called  the  lethul  area.  A[  and  Core  calculated  under 
the  assumption  of  u  uniform  target  distribution.  As  to  multiple  volleys,  the  assumption 
is  mude  that  the  percentage  reduction  g  in  F  will  be  directly  proportional  to  the  respec¬ 
tive  g„  in  F(l  (n  volleys).  Through  the  use  of  this  methodology,  Spears  strives  to  arrive 
at  the  conclusion  that  "Changes  in  single-volley  Coverage  of  a  target  by  a  weapons  ef¬ 
fects  pattern  (a  quantity  relatively  easy  to  determine)  can  be  used  us  a  basis  for  deter¬ 
mining  eriticul  reductions  in  effectiveness  of  multi-volley  fire  (a  quantity  difficult  to 
determine  accurately).”  Through  the  introduction  of -S—  S  Ay  as  a  measure  of  the 

■^p  i 


I.  Numbers  in  parentheses  appearing  in  the  text  refer  to  “LITERATURE  CITED,” 
p.  13,  while  numbers  in  parentheses  on  the  right  margin  refer  to  equations. 
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average  friiiiiun  of  (lie  firepower  whirl)  hits  I  hi*  target.  i.e..  I  In-  tolal  casualty  potential 
mlnn-il  lis  I  hr  farlor  “riiiiiiiiiin  area  between  target  anil  effects  pattern  divided  hy  lln- 

•  •  »  «•  .  i  ■  I-  .  -I  il  il  V  4  /  4  1 1  . 

MliTh  |  fill  I  rm  ana  ami  iim*  iii  ui«*  i  iiimmiii  uimi  iimiimhi  mui - ~  **  *jy  ■  v|  m  hm 

exponential.  Spears  arrives  al  a  new  niiii-farlorizi'il.  or  ini\rcl.  K.  This  derivation  is, 
however,  not  | n -r m is> i I >l<-  and  conflicts  accordingly  with  the  former  result  I-  -  f  •  <..  At 
In-si.  it  is  an  a|i|iro\irnalion  within  an  a|i|iro\imale  framework.  Heferenee  (4)  stales 
that  the  assumption  of  proportionality  In-tween  casualties  anil  fraetional  coverage  is 
the  weakest  link  in  the  methodology  employed  in  Kef.  (1)  and  (2)  and  eriticizes  various 
other  assumptions  made.  I  rider  the  eriterion  that  the  target  aeipiisilion  does  not  de¬ 
grade  the  aeeuraey  of  the  vvea|>oiis  systems  itself  by  more  than  HY/r  and  under  the  as¬ 
sumption  that  the  map  aeeuraey  or  error  is  the  prim  ipal  eontrihiitor  to  the  weapon  site 
survey  error  and  the  target  loeation  error,  it  is  concluded  in  Ref.  f4)  that  present  map 
iieriiraeies  ran  Ih-  relaxed  or  that  the  (llass  A  National  Map  Standards  have  ahoiil  twiee 
the  required  precision.  This  result  lias  been  obtained  hy  simple  ealeiilatioiis  based  oil 
the  assumption  that  the  total  varianee  is  the  sum  of  the  individual  variances  including 
(lie  geodetic  one  (target  location  error).  This  assumption  is  wrong  and  is  one  of  the 
basic  weaknesses  of  all  analyses  so  far,  apart  from  a  rather  primitive  methodology . 

1.4  In  view  of  the  shortcomings  enumerated  in  paragraph  1.2  and  in  order  to 
provide  a  sound  liusis  for  decision  making,  this  report  was  undertaken.  Objectives  of 
the  report  arc  as  follows: 

1.4.1  A  rigorous  mathematical-statistical  analysis  involving  a  direct,  physi¬ 
cal  approach  to  the  priihlrm.  In  particular,  this  analysis'shull  Ih-  independent  of  as¬ 
sumptions  regarding  target  distributions  and  simplifications  involving,  c.g..  proportion¬ 
ality  between  casualties  and  fractional  coverage,  and  casualty  potential. 

1.4.2  Inclusion  of  multiple  shots  and  multiple  volleys  without  loss  of  rigor 
or  generality . 

1.4.4  Optimization  for  multiple  volleys  us  a  new  and  most  significant  dis¬ 
covery. 


1.4.4  donsidcratiori  of  inhomogeneous  target  distributions  and  its  change 
after  the  first  volley. 

1.4.4  Incorporation  of  inetcorologii-al -error  variances. 

1.4.6  l  lilizuliou  of  non  circular  distribution  parameters. 

1.4.7  Application  of  ijoii-isotropic  fragmentation  patterns. 
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I.4.H  A  total  .-It'titri  optimization  or  marginal  utility  analysis  involving 
tlir  whole  range  and  employment  spectrum  of  a  weapons  systems,  i.r.,  a  grand  optimum. 

1.5  A  contemplated  Part  II  of  this  study  will  include  a  supplemental  analysis  fur 
height  hursts  (time  and  ambient  fuzes)  including  vertical  target  location  errors  -  Parts 
III  and  IV  will  deal  with  eratering  conventional  weapons  and  nuclear  weapons  respec¬ 
tively,  and  a  partially  different  methodology  will  be  required  in  these  areas. 

1.6  The  optimal  aiming  pattern  analysis  together  with  the  optimal  overall 
weapons-systems  employment  concept  developed  in  this  report  allow— on  the  average- 
a  considerable  relaxation  pertaining  to  stringent  target-acquisition  requirements  in  gen¬ 
eral  and  map-accuracy  requirements  in  particular.  They  tend  to  shorten  the  firing  en¬ 
gagement  time  and  are  also  advantageous  in  case  of  ammunition  shortage.  An  excep¬ 
tion  would  be  hardened-point  targets.  According  to  the  experience  gained  (as  men-, 
tinned  in  footnote  4).  target-locution  errors  can  bo  very  large,  and  identification  and 
location  problems  will  probably  exist  for  longer  distances  if  direct  distance  and  azi¬ 
muth  measurements  are  performed,  though  to  a  lesser  extent.  Meteorological  errors 
arc  also  not  supposed  to  become  negligible  under  many  combat  conditions.  In  view  of 
the  above,  numerically  fixed  and  extreme  accuracy  requirements  synonymous  with 
sophisticaied  and  very  expensive  equipment  which  very  often  does  not  live  up  to  ex- 
|K‘ctulions  under  realistic  conditions  are  unnecessary.  The  R&D  process  in  the  areas  of 
more  accurate  mapping  and  target  location  being  essentially  independent  of  that  per¬ 
taining  to  new  weapons  systems  should  he  pursued  at  u  normal  technological  pace  and 
should  not  overemphasize  accuracy  but  rather  concentrate  on  versatility,  reliability, 
and  survivability.  This  is  also  consistent  with  a  recent  directive  of  the  Army  Chief  of 
Staff. 


Ah  exhibited  by  this  study,  the  intru  and  extra  weupons  systems  employment 
parameters  are  interdependent,  variable,  and  changing  but  nevertheless  allow  a  continu¬ 
ous  integrated  operational  optimization.  In  so  far,  the  study  is  also  of  significance  for 
the  Ceographic  Intelligence  and  Topographic  Support  Systems  Study  ((1IANT),  the  de¬ 
velopment  of  the  Position  and  Azimuth  Determining  System  (PADS),  and  the  develop¬ 
ment  of  the  l.ong  Range  Position  Determining  System  (I.KPDS).  Finally,  the  methods 
outlined  run  serve  as  a  research  and  weapons-systems  analysis  tool  for  both  Hie  Comlial 
Developments  Commund  and  the  Materiel  Command. 


In  this  regard,  it  may  lie  worthwhile  (o  mention  that,  according  to  Ref.  (il)-O.  S, 
Speurs.  Scientific  Advisor  to  CI)C.  Artillery  Agency,  has  stated  the  following:  "ll 
is  not  that  we  don't  consider  (he  vertical  component  important,  we  simply  realize 
that  it  is  a  difficult  problem  to  solve.  Once  we  gel  a  complete  handle  on  the  hori¬ 
zontal  accuracies,  vve  will  lie  aide  lo  slarl  tackling  I  hat  problem  more  inlclligcnllv 
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2.  Individual  Hit  Probabilities.  The  fragment  damage  pattern  of  a  parlieular 
artillery  shell  is  not  isotropie  as  ean  he  inferred  from  Fig.  I.  It  depends  also  on  range, 
height  ot  burst,  and  fmpael  angle.  Tabulations  eoniairi.  in  general,  isoiropie  daia  in- 
eluding  distance  from  burst,  total  number  of  effective  fragments,  and  average  number 
of  effective  fragments  per  area  unit. 


An  example  from  Kef.  (6)  is  given  below: 


Fragment  Damage  of  Shell.  IIK,  155-mm.  Ml 07: 
Initial  Fragment  Velocity  3,500  f/s 
Source:  Army  TIM  9-1907,  Table  XXXV 
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770 
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.00032 

.235 

503 

700 

420 

.00007 

.515 

340 

From  individual,  i.e.,  unaveraged,  fragment  patterns,  it  is  possible  to  determine 
through  the  use  of  sampling  techniques  individual  hit  probabilities,  Thus,  pj  (F,  p.  4>) 
would  be  the  average  probability  that  a  |»erson  or  item  with  cross  section  F  which  is 
located  at  a  distance  p  and  azimuth  0  from  the  burst  suffers  exactly  one  hit.  In  this 
respect,  the,  azimuth  is  to  be  counted  counterclockwise  from  the  line  of  fire.  By 
pj  (F,  p,  0)  =  pj  +  Pj  +  ....  wc  designate  the  probability  of  at  least  one  hit.  With  refer¬ 
ence  to  human  beings,  it  would  be  possible  to  drop  the  letter  F.  For  identification 
purposes,  we  denote  a  semi  fixed  pattern  of  human  beings  by  superscripts  and  have 


3.  An  excellent  introduction  into  kinds  and  characteristics  of  explosives  (munitions) 
is  given  in  Ref.  (7). 
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thus  for  example,  p 
refer  !-.: 


|  ®  (p.  <p)  p  |  (P|4,.  0|.j.)-  It  is  •"  be  kepi  in  mind  that  these 
s!’**!  r:*. iwiglii  of  hurst.  and  impact  angle. 


proh- 


3.  Distributions  and  Distribution  Parameters.  Weapon  distribution  parameters 
for  a  spec i fie  range  are  the  line  of  fire  and  lateral  standard  deviations  Sf  and  S^.  As  al¬ 
ready  mentioned,  height  uncertainties  are  considered  negligible  in  this  investigation 
(Part  I).  In  addition,  we  have  target  donation  errors  depending,  e  g.,  on  map  accuracy, 
target  identification,  and  location.^  and  meteorological  errors.  The  corresponding  dis¬ 
tributions  may  for  simplicity  be  described  by  the  two  parameters  o-p  and  Oyj.  Since  we 
restrict  ourselves  to  normal  distributions,  we  may  establish  the  relation 


var  r-p  +  van  r^j  =  var  fi-p  +  var 


=.o^ 


It  is  important  to  remember  that  0“  and  o“^  need  not  he  considered  constant  for  a  cer¬ 
tain  range,  lienee,  o~  may  allow  a  few  classes  of  variability  depending  on  eireumstanees. 


4.  Formulation  of  Multiple  Volley  Optimization  Problem,  for  Stationary  Per-  / 
gonnel  Distribution.  In  Fig.  2,  the  general  target  coordinate  system  for  aiming  purposes 
is  denoted  by  x,  y.  At  the  origin,  the  combined  target  location  distribution  / 


f(£ V)  £  dr?  =  — ~Tj 
2iro~ 


d  {  d  rj 


has  a  maximum.  The  aiming  point  for  the  firsl  volley  is  represented  by  with  co¬ 
ordinates  U|,  hj ;  and  the  respective  gun-aiming  points  separated  by  the  distance  e  are 


4.  The  accuracy  of  a  class  A  map  of  1/50.000  scale  within  a  single  sheet  cun  be  ex¬ 
pressed  approximately  by  a  standard  deviation  of  2-' lit.  Though  Ibis  is  not  a  neg¬ 
ligible  parameter  and  accuracies  decrease  with  reference  to  lower  quality  maps, 
additional  errprs  enter  in  ease  of  target  identification  (which  includes  determina¬ 
tion  of  a  reference  point  for  the  whole  target  configuration)  and  large!  location 
on  the  map.  The  latter  type  of  error  can  he  very  sizable,  and  standard  deviations 
of  the  order  500m  have  been  found  according  to  Ref,  (H),  (9),  ami  (10).  For  sim¬ 
plicity.  we  lump  map,  target  identification  and  large!  location  variances  together 
into  var  r-p.  Smaller  o-p  "s  arc.  of  course,  to  be  expected  in  ease  of  a  direct  link  in¬ 
cluding  distance  and  azimuth  measurements  between  observer  and  a  suitable  tar¬ 
get  reference  point.  Diioetional  Op's  might  a'*o  be  generated  by  moving  targets 
and  target  configurations. 


”’85 


Line  of  Fire 


l  «v 
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The  general  target  coordinate  system. 


C.|,  G2,  c;3,  c;4.  The  burst  point  for  which  a  total  impact  probability  is  to  be  comput 
cd  and  which  lies  in  a  finite  area  element  (for  numerical  purposes)  is  It.  Only  one  indi¬ 
vidual  target,  T|,  is  indicated  within  the  target  area  with  boundary  T.  The  distance 
from  It  to  T|  is  p  and  the  azimuth  is  <p  commensurable  with  the  denotations  of  para.  2. 

for  the  first  volley,  we  have  lour  burst  distributions  designated  by 


X|(\.v:  a|+£  ,  bj+^  :  Sr,Sj)dxdv.  X^x.y :  aj  bj+^  :  Sr,Sg).  etc. 


(3) 


For  sufficiently  small  area  elements  Ax  Ay.  we  arrive  then  in  integral  form  at  an  inter¬ 
mediate  average  probability  of  hitting  Tj  at  least  once. 


+0O 


+  00 


Pjl  =  J/EXV  '  p!(p,0)dxdy  =  jj  E  Xv  p]  (x,  y,  £,  t?  tj,)  dxdy 


(0 


and,  since  O3  obeys  a  distribution  law,  at 
+00  +00  | 

I 

1 


TV"*  ^ 

i’J =jj  ff  Pi(*'M’fliti*ni)f(£in)dxdyd£dT? 


(5) 


For  k  volleys  and  p  individual  targets,  we  obtain  the  total  expected  casualty  result 

+00+00  . 

r=k  p=N  r  r  r  f  v=4 

n.  k  -  E  E  H  I  E\v(x,yair,br)pj(x,y;i,Jj;f„,rj„)f(£,^)dxdyd£d^  (6) 

't  .  r=lp=l  JJ  J J  v=l 

-00  -00 


The  optimization  conditions  can  be  formulated  as 


n  k 


-  0 


JL 

3b, 


k 

2 

1 


=  0 


C) 


For  a  circular  and  homogeneous  (uniform)  target  distribution  conditions  (7)  reduce  to 
fewer  equations,  i.e.,  the  respective  aiming  pattern,  consisting  of  a  set  of  k  origins  0>> 
would  be  invariant  under  a  rotation  ubuut  Op 

After  the  first  volley  which  has  in  many  cbbcs  a  surprise  effect,  a  degradation  with 
respect  to  Pj  can  be  expected  which  can  be  expressed  as  a  transition  from  ground  to  air 
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burst'’  ami  by  an  empirical  reduction  factor  R  <  1.00.  Taking  this  into  consideration, 
a  more  general  result  corresponding  to  eq.  (6)  reads  as 

+  OO  +  £30 

ff  ff  '  4 


n  t  :  i, 


H  \  PI :  i  n:  {  ( ,  ,  ))f({.i?)dxdyd£di7 


r=  h^>=N  }?}?& 


■ •»  r  •  ^  x 

+  E  L  K  J^%(2)P|C(x>>;f.i?;^(2).^(2))f(?.q)dxdyd?drj 

2  ■  i  ,  , 


In  eq.  (R),  the  first  term  refers  to  the  first  volley.  The  second  term  containing  the 
reduction  factor  R  reflects  a  changed  hit  probability  function  and  includes  the  factor 

V2> 

A  particular  ir„(2)«  sav  ir  ,2)’  requires  the  computation  of  the  individual  P**  from  eq. 
(5).  It  is  then  3  1 

I  -  k  P?  (9) 


where  k  denotes  the  (average)  probability  that  an  individual,  hit  at  least  once,  remains 
at  the  initial  position.  The  index  (2)  in  *  /2\  indicates  the  transition  from  the  initial 
target  configuration  to  a  second,  more  protective  one. 

It  should  he  mentioned  that,  in  connection  with  an  evaluation  of  eq.  (8),  an  aver¬ 
age  iv  j  for  typical  lurget  distributions  under  consideration  of  protective  obstacles  can 

Ik-  determined.  It  is  also  possible  to  classify  targets  by  size  and  concentration  indices 
(cf.  para.  7).  Furthermore,  it  is  possible  to  split  P|  up  into  probabilities  for  exactly  I 
hit,  2  hits,  etc, 

5.  Probability  Distributions.  The  probability  distribution  associated  with  n  k 

njS  1 

of  eq.  (tt)  ean  easily  (though  approximately)  be  found  by  setting  p  =  — -  where  N 


f>.  According  to  Ref.  (6).  p.  IHI,  ground  bursts  generally  are  more  effective  against 
material  and  personnel  in  case  of  no  shielding  by  revetments,  but  personnel  in  fox¬ 
holes  or  trenches  should  be  attaeked  by  uir-burst  fire. 
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is  I  In*  total  tin  mi  Imt  of  iinli\iiliiiil  targets.  According  lit  Krmlall  (II).  mi*  haw*  thru  till* 
moments  alimit  llir  inrun  Np  ^ 

fj.,  N|ii|  7j  -  1  -  p 

P.j  W,  I*) 

P4  -  3  N"  f»"«r +  i»  «i N  ( 1  ftp  ri>  ' 

from  m  liiili  a  Pearson  Ty  («■  I  curve  i-an  In-  calculated  which  is  cwduuhlc  in  ti-rms  of  I  hr 
incomplete  It-funrlion.  A  lti*rnnl i\i-l\ .  I  hr  binomial  ilistrilmilion  anil  associated  tables 
might  III*  used. 


6.  Symmetric  Aiming  Pat  terns.  \s  an  illustration,  some  sy  mmetric  aiming  pat- 
li-rns  a|i|iliealile  for  circular  anil  complcti-ly  homogeneous  eondilions  are  shown  in  l-'ig. 
3.  l-.i 1 1  ii v oli-ti I  solutions  Mould  result  through  an  arbitrary  rotation. 


k  -  l 


k  -  2  k  •  3 
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k  *  4  k  ■  5 


lig.  it.  Symmetric  aiming  patterns. 


7.  Computation  and  Utilisation  Considerations.  A  purlieular  kind  of  quasi- 
eireiilar  target  could  profitably  though  not  exclusively  be  characterized  by  r  (range  to 
origin  0|)  S  (target  size.  3  indices),  I)  (quadrant  density,  3  indices),  a  (combined  tar¬ 
get  location  error.  3  or  4  indices),  k  (number  of  volley  s).  From  them!  data,  k  azimuth 
and  angular  height  corrections  for  I  lie  optiinul  aiming  points  would  be  immediately 
available.  Of  course,  the  optimization  computations  should  Ik*  conducted  by  a  large- 
-vale  digital  computer,  i.e..  no  I  in  the  field.  The  corrections  -  functions  of  variable 
in|iul  data  would  lie  available  us  stored  digital  information  ui  .4  incorporated  in  TAC- 
FI II F.  procedures.  Probability  si ulemenls  depending  on  k  could  he  added.  Too  many 
indices  arc  to  he  avoided.  As  to  I).  there  are  3*^  variations  (with  three  elements  of  the 
Fourth  class  and  repetitions).  Some  of  these  variations  run  be  omitted  because  of  prac¬ 
tical  reasons.  Nine  quadrants  leading  to  3^  complexions  would  be  prohibitive.  From  a 
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scientific  standpoint,  i-  no  appears  that  an  effective  employment  of  a  single  artillery 
weapons  system  (battery)  represents  a  rather  formidable  problem. 

H.  Views  on  Optimal  Weapons  Systems  Employment. 

8.1  With  reference  to  optimization  considerations,  we  assume  the  existence  of 
the  following  scheme: 

rl  hllllM<kl|)  .  ,lln l' In^ln^  1 


r2  h2lll21<k2|)  .  h2nl!2ri^2n^ 


*'ml  *  ml 


*'mn  ^  mu 


In  this  discrete  scheme,  valid  for  a  particular  weapons  system  (e.g.,  esullery  battery), 
the  symliols  r,  h.  I',  and  k  denote  range,  relative  frequency  of  employment,  mean 
weupons  effect  for  a  particular  type  of  target,  and  number  of  volleys  respectively. 
Strictly  speaking,  there  has  to  be  a  greater  number  of  diserctc  schemes  with  associated 
scheme  frequencies  in  order  to  account  for  variations  in  target  size  and  target  location. 
This  involves  un  additional  frequency  matrix  with  elements  ja  j,.  The  total  mean  weap¬ 
ons  systems  effect  can  then  be  formulated  as 

l  =  *'a,0  ^  ia.b^a.0;a.b^a.0sa,b)  (**) 

„  /x  A 

We  shall  distinguish  lietween  l1  and  U  with  U  considered  optimized  by  the 
unulysis  outlined  in  para.  4.  In  other  words,  C  does  not  imply  the  utilization  of  opti¬ 
mal  uiming  patterns. 

If  we  apply  the  rather  usual  criterion  of  MY/r  damage  or  casualties  with  a  WA 
assurance,  we  arrive  at 

•  K "  ^  u£t,  ka^:«.h  <  12) 


MY.rmY.r 


h  -  £  I  k 


a  (S  a  n  h®*P  ;  “■l> 
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with  (I  *  l:  ami  k  <  k. 


A  reasonable  measure  for  the  effeetiveness  increase  expressed  in  [HTeent  is  evidently 


n  =  i(H»  K  -JL 

k 


(14) 


8.2  A  different  approach  would  consist  of  stipulating  a  constraint,  say 


and  to  compute  the  k^'s  in  sueli  a  way  that 
W2>  -  Max. 


(15) 


(16) 


For  the  purpose  of  comparison,  we  may  assume 

K<2)=K  (17) 

The  optimi'/.ution  expressed  by  eq.  (15)  and  (16)  implies  a  fortvriori 

C<2>  >  0  (18) 

and  a  relatively  greater  expansion  of  volleys  with  respect  to  closer  range  targets  and 
those  involving  smaller  a's.  On  I  he  other  hand,  for  some  turgets  with  less  favorable 
characteristics,  the  30%/90%  criterion  might  not  be  fulfilled.**  What  can  be  said  with 
certainty  is  that  the  utilisation  of  optimal  aiming  patterns  makes  the  ground  optimiza¬ 
tion  described  in  para.  8.2  quite  attractive.  It  is  conservatively  estimated  that  optimal, 
aiming-pattern  utilization  incorporated  in  TAflFIKF,  would  result  In  u  15 %  increase  in 
systems'  effectiveness.  The  systems'  overall  optimization  would  yield  an  additional 
15%  increase  and  thus  lead  to  a  combined  improvement  of  30%. 


This  is,  however,  not  a  serious  limitation  since  it  cun  be  partiully  or  completely 
overcome  by  a  greater  A  in  eq.  (15).  This  would  particularly  apply  to  defensive 
positions  with  a  large  ammunition  supply. 
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ABSTRACT 

This  paper  proposes  a  new  and  unique  approach  for  conducting  comparative  experiments 
or  evaluations  betv/een  existing  or  proposed  air  defense  weapon  systems.  It  is  based  upon 
the  game  theory  "minimax"  philosophy  and  provides  several  distinct  advantages  over  the 
use  of  computer  simulation  methods.  Submodels  for  objectively  determining  the  optimal 
deployment  of  the  defense  and  the  optimal  attack  routes  to  be  used  by  the  attacking  air¬ 
craft  are  discussed. 


INTRODUCTION 

Development  and  deployment  of  air  defense  systems  having  a  large  degree  of 
effectiveness  against  high  altitude  aircraft,  has  resulted  in  Increased  interest  In  the  opera¬ 
tion  of  tactical  aircraft  at  low  altitudes  111.  As  a  result  many  weapons  systems  analysts 
have  become  deeply  Involved  in  analytical  and  experimental  studies  evaluating  the  effec¬ 
tiveness  of  existing  or  proposed  defense  systems  for  defeating  the  low  altitude  threat. 
Historically,  war  games  have  been  extensively  used  to  "model"  militury  situations  for 

Thie  article  hae  been  reproduced  photographically  from  the  author*'  manuscript. 
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such  experimental  and  evaluative  purposes.  The  different  types  or  classes  of  war  games 
that  have  been  used  are:  (1)  field  exercises,  (2)  board  games,  and  (3)  computer  simula¬ 
tions. 

When  considering  air  to  ground  conflict  situations,  experimentation  using  the 
computer  simulation  technique  has  proven  to  be  the  most  feasible  and  efficient.  Conse¬ 
quently,  computer  simulation  models  have  evolved  from  very  simple  and  basic  models  into 
those  which  are  now  very  targe,  complex  and  time  consuming.  This  increase  in  size  and 
complexity  has  arisen  due  to  the  desire  to  approach,  as  near  as  practically  possible,  an 
exact  model  of  the  real  life  situation.  Unfortunately,  as  realism  has  increased,  so  too 
has  the  computer  time  required  to  run  the  experiments. 

This  paper  proposes  a  new  approach  for  conducting  comparative  experiments  or 
evaluations  between  existing  or  proposed  air  defense  weapon  systems.  It  is  based  upon 

■I 

the  game  theory  "minimax"  philosophy  and  provides  several  distinct  advantages  over  existing 

computer  simulation  models.  These  include: 

I ' 

1.  Less  computation  time  required. 

2.  Fewpr  necessary  assumptions  and  simplifications,  hence  greater 
realism. 

3.  Additional  useful  information  is  generated  such  as  optimal  defense 
system  deployment,  optimal  attack  routes,  etc. 

4.  Real  world  scenarios,  (actual  situations)  maybe  used. 

5.  Only  one  computer  run  per  defense  system  is  required. 
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MODEL  PHILOSOPHY 


The  scenario  used  For  the  Air  Defense  Comparative  Mode!  should  be  u  reui  wui'u 
situation  i.e.  a  specific  piece  of  terrain  which  is  to  be  defended.  The  concept  of  the 
low  altitude  attack  is  to  utilize  the  masking  effects  of  the  terrain  (hills,  valleys,  etc.) 
and  of  the  earth's  curvature,  to  prevent  the  defense  from  being  able  to  detect  and  engage 
the  attackers  until  the  targets  are  reached  ill.  Hence,  any  experimental  evaluation 
of  the  defense  system  must  take  this  into  account. 

The  defensive  problem  in  our  scenario  can  be  stated  as  follows: 

1.  Given  a  specific  sector  of  terrain  (with  hills,  valleys,  etc.)  which 
is  to  be  defended  by  n  or  less  defensive  units. 

2.  Given  the  characteristics  of  the  defense  system  (i.e.  range,  maximum 
and  minimum  elevation  angles,  azimuth  scan  angle,  etc.). 

3.  Given  the  feasible  locations  and  pointing  angles  for  the  placement 
of  defensive  units.  (I.e.  cannot  be  located  in  middle  of  lakes, 
bottom  of  ravines,  etc.), 

4.  Find  those  n  locations  and  pointing  angles  which  (a)  minimize 
the  range  from  any  attacker  to  a  systems  radar  and  (b)  maximizes 
the  visibility  of  the  combined  radar  systems.  This  must  take  into 
account  the  masking  effects  of  terrain  features  and  earth's  curva¬ 
ture. 

Likewise,  we  can  state  the  problem  faced  by  the  offense  or  attackers  in  our 
scenario.  This  can  be  done  as  follows: 
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.  Given  a  set  of  targets  to  be  destroyed  which  are  contained 
uj rt  cp piece  c f  * c rr^. lr.  '.v h .  c  h  defended  by  c  set 

of  ground  based  air  defense  missile  systems  which  are  optimally 
deployed. 

2.  Given  that  the  attacker  or  penetrator  has  complete  knowledge 
from  intelligence  operations  of  these  allocations  and  of  the 
defensive  capabilities. 

3.  Determine  the  best  location  to  enter  the  defended  sector,  and 
then  the  least  risk  route  to  follow  in  order  to  reach  a  designated 
target.  The  least  risk  route  is  that  which  minimizes  the  visibility 
time  and  maximizes  the  survival  probability. 

f 

In  the  proposed  Air  Defence  Comparative  Model,  these  problems  are  solved  ob- 

l  ’ 

jectively  and  optimally  by  sub-models.  The  objective,  optimal  solution  to'both  problems, 
is  a  unique  feature  of  the  proposed  model.  The  tactics  are  not  determined  by  educated 
guess  os  in  other  war  game  models.  It  should  be  pointed  out  however,  that  the  two  sub¬ 
models  (optimal  allocation  of  defense  units  and  optimal  attack  route  analyzer)  can  be 
used  to  set  the  scenario  and  tactics  for  other  computer  simulation  models.  It  is  a  firm 
conviction  of  the  authors,  that  where  tactics  are  determined  by  educated  guess,  the 
experimenter  may  inadvertently  penalize  a  system  by  his  choices.  Allocating  or  placing 
the  defensive  units  by  the  use  of  the  optimal  allocation  model  on  the  other  hand,  allows 
each  different  system  to  capitalize  on  its  strengths  and  minimize  its  weaknesses. 

The  concept  of  the  proposed  comparative  model  can  now  be  stated.  The  philos¬ 
ophy  folio-1.' :d  is: 
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1.  Determine  the  optimal  defense  system  deployment  for  each  system 
to  be  considered,  based  upon  its  own  characteristics  and  the 
terrain  features  of  the  sector  to  be  defended. 

2.  Determine  the  optimal  attack  routes  against  each  defense  system 
which  minimizes  the  risk  to  the  attacking  aircraft. 

3.  Determine  the  risk  incurred  by  the  attacker  For  each  defense  system 
to  be  compared . 

4.  The  defense  syjtem  that  maximizes  the  enemy's  risk  is  the  preferred 

* 

system. 

Maximization  of  risk  to  the  enemy  has  been  chosen  as  the  measure  of  effective¬ 
ness  for  a  very  straightforward  reason.  The  purpose  of  the  air  defense  system  is  to 
protect  field  army  value  units  such  as  supply  depots,  vehicle  concentrations,  artillery 
positions,  troop  concentrations,  etc.  The  purpose  of  any  offensive  weapons  system  is 
to  destroy  a  given  set  of  targets  (value  units)  with  the  least  possible  cost.  It  is  a  generally 
accepted  fact  that  a  defensive  system  cannot  prevent  a  determined  and  powerful  offense 
from  destroying  a  given  number  of  these  targets  if  the  offense  is  willing  to  pay  the  price. 
The  defense  objective  then  is  to  try  to  extract  a  high  cost  from  the  offense.  In  gaming 
theory  terms  then,  the  goal  of  the  defense  is  to  maximize  the  offensive  cost  while  min¬ 
imizing  the  defensive  cost.  Both  offensive  and  defensive  costs  are  direct  functions  of 
the  risk  incurred  by  the  offense  in  carrying  out  its  attack. 

The  proposed  Air  Defense  Comparative  Model  is  a  Game  Theoretic  Model  utilizing 
the  maxi-min  principle  of  optimality,  [2]  Stated  simply  the  defense  chooses  that  strategy 
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which  maximizes  minimum  risk  while  the  offense  chooses  that  strategy  which  minimizes 
maximum  risk.  The  value  of  the  game  is  then  calculated  for  each  defensive  system  to 
be  compared  and  the  one  which  extracts  the  highest  risk  to  the  attackers,  is  the  pre- 
ferrable  sys*em. 

An  overall  schematic  of  the  model  is  shown  in  Figure  1.  Due  to  space  limitations, 
it  will  not  be  possible  to  give  detailed  descriptions  of  the  sub-models  in  this  paper.  How¬ 
ever  detailed  descriptions  of  the  component  sub-models,  including  computer  programs,  1 
may  be  found  in  references  3,  9,  10,  and  11.  Short  descriptions  of  the  sub-models  are 
given  in  the  following  sections. 

MAVD  MODEL 

Basic  to  the  proposed  Air  Defense  Comparative  Model  is  the  visibility  subroutine 
called  MAVD  (Minimum  Altitude  Visibility  Diagram).  MAVD  is  a  new  concept  and  sub¬ 
routine  for  calculating  the  visibility  of  targets  to  the  defensive  system  sensor  units  [3l. 

The  input  to  the  MAVD  Model  is  an  array  of  digitized  topographic  data  which 
is  stored  on  magnetic  tape.  The  Arrny  Map  Service  has  expended  a  considerable  amount 
of  time  and  effort  in  the  digitization,  ond  storing  on  magnetic  tape,  of  topographic  data. 

An  mx n  grid  of  horizontal  (m  =  1 ,2, . . .  ,i)  and  vertical  (n  =  1 ,2, , . . , j)  lines  is  overlaid 
over  the  topographical  map  of  the  piece  of  terrain  of  interest.  The  spacing  or  grid  interval 
between  the  lines  is  equal.  The  standard  army  battle  map  is  a  transverse  Mercator  projection 
of  the  Gauss-Kruger  type  [4] .  The  primary  coordinate  system  for  the  map  is  a  square  grid 
system  called  the  Universal  Transverse  Mercator  grid  [5].  Points  of  interest  con  be  located 
on  the  map  by  theii  UTM  g"  id  coordinates.  The  UTM  grid  will  appear  on  any  map  as  a 
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square  grid  system  where  the  numerical  values  of  the  coordinates  of  a  point  ore  positive, 
and  increase  as  one  moves  the  point  east  and  north.  For  good  terrain  definition,  it  has 
been  found  that  the  grid  spacing  should  not  exceed  about  1,500  feet  or  300  meters.  The 
local  altitudes  above  sea  level  for  the  grid  points  thus  define'd  are  read  off  the  topo¬ 
graphical  map  and  entered  along  with  their  grid  point  designation  (i,j)  as  inputs. 

MAVD  (Minimum  Altitude  Visibility  Diagram)  is  a  geographic  representation  of 
the  minimum  local  altitude  at  which  a  target  may  fly  above  the  local  terrain  and  still  be 
visible  to  the  given  air  defense  sensor.  Thus  a  MAVD  value  of  150  feet  at  point  5,  45 
(the  i,j  grid  representation  of  a  specific  point  on  the  terrain)  meons  that  any  aircraft 
at  150  feet  altitude  or  above  is  visible  to  the  sensor,  or  conversely  any  aircraft  below 
150  feet  altitude  is  not  visible  (either  masked  by  terrain  irregularities  or  the  curvature 
of  the  eaith)  to  the  sensor. 

The  MAVD  routine  is  used  to  compute  all  the  MAVD  values  for  every  designated 
point  (i.e.,  a  point  defined  by  the  Intersection  of  two  grid  lines)  on  a  grid  for  all  given 
sertsor  locations.  Figure  2  represents  an  example  of  a  MAVD  display.  The  top  figure 
(2o)  is  the  original  terrain  map  and  the  bottom  figure  (2b)  is  the  MAVD  display  where 

each  MAVD  value  is  given  for  the  corresponding  point  pr>  the  original  terrain  map.  In 

i  / 

three  dimensions  a  surface  through  all  the  MAVD  values  could  be  represented  and  ony 
aircraft  on  or  above  this  surface  is  visible  to  the  given  sensor(s). 

The  values  on  the  MAVD  represent,  as  mentioned,  the  minimum  altitude  values 
at  which  a  penetrator  is  visible  to  a  sensor  at  any  given  grid  point  (intersection  point 
represented  by  the  intersection  of  an  "I"  and  "j"  line).  The  effect  of  the  curvature  of 
the  earth's  surface  and  all  terrain  irregularities  arc  considered  in  the  computation  of  these 
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values.  The  calculation  procedure  is  straightforward  and  uses  basic  plane  and  analytical 

TL«  AAmMufAr  MrAnrnm  f#>r  tUi*  «i i^rrv »f !na  /u/rlUfln  in  pArtrnn 
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IV)  may  be  found  in  reference  3.  The  present  program  is  capable  of  handling  a  21 1  x211 
grid  size. 

DEFENSE  ALLOCATION  MODEL 

The  second  sub-routine  utilized  is  the  Allocation  program  which  provides  a  system¬ 
atic,  objective  method  for  computing  the  optimal  deployment  of  any  air  defense  system. 
Allocation  is  defined,  with  respect  to  this  paper,  as  the  assignment  (or  placement)  of  air 
defense  system  sensors  at  specific  points  on  the  given  piece  of  terrain.  The  optimal  allo¬ 
cation  is  that  deployment  which  maximizes  the  attacker's  risk.  It  may  be  also  thought  of 
as  that  deployment  which  minimizes  the  probability  that  an  attacking  aircraft  or  missile 
penetrates  the  defensive  system  undetected. 

A  survey  of  the  literature  uncovered  an  almost  negligible  amount  of  effort  towards 
devising  any  systematic,  objective,  assignment  of  sensor  units  to  terrain.  The  majority 
of  models  surveyed  assigned  sensor  locations  at  random  or  at  best,  use  an  educated  guess 
based  on  an  "analysis"  of  the  terrain  involved.  This  analysis  consists  of  little  more  than 
looking  at  the  terrain  map  and  attempting  to  visualize  the  effect  of  placing  a  sensor  at 
a  certain  point.  Such  methods  of  choosing  sensor  locations  are  for  from  optimum.  It 
is  highly  subjective  and  consequently  it  is  doubtful  that  any  two  people  would  choose 
the  same  locations. 

The  mathematical  formulation  of  optimal  deployment  problems  falls  into  a  sub-, 
class  of  non-linear,  zero-one  programming  problems.  Although  this  has  been  previously 
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recognized,  [6]  it  has  not  been  possible  to  apply  the  existing  methods  of  zero-one  pro¬ 
gramming  to  any  practical  size  problem  due,  to  the  severe  limitations  of  these  mathematical 
methods.  A  new  procedure  called  Complementary  Programming,  was  therefore  developed 
as  a  part  of  this  research  [7]  and  is  applicable  to  very  large  problem  types.  For  example, 
it  was  used  to  compute  the  optimal  deployment  of  radars  within  the  continental  United 
States.  The  results  were  then  compared  with  those  previously  proposed  by  Smallwood  [8] 
utilizing  a  much  more  involved  and  time  consuming  procedure.  The  Complementary  Pro¬ 
gramming  method  achieved  an  inproved  deployment  over  Smallwood's  "optimal  method." 

Tests  conducted  during  the  evaluation  of  an  early  version  of  the  Allocation  Pro¬ 
gram  showed  that  optimal  deployment  was  sensitive  to  both  range  and  visibility.  The  tests 
showed  that,  for  a  large  terrain  area,  the  primary  factor  In  deployment  was  range,  and 
Visibility  was  only  secondary.  This  observation  led  us  to  divide  the  original  allocation 
program  Into  two  separate  programs.  We  have  designated  the  first  program  as  the  Coarse 
Allocation  Program  and  the  second  as  the  Fine  Allocation  Program.  The  main  concern  of 
the  Coarse  Allocation  Program  is  the  minimization  of  range  distance  while  the  purpose  of 
the  Fine  Allocation  Program  is  the  maximization  of  radar  system  visibility.  The  two  programs 
are  then  used  s^uentially  (see  Figure  3).  A  good  analogy  to  this  method  is  the  process  used 
in  turning  to  a  station  on  the  radio.  One  first  turns  the  selector  to  the  vicinity  of  the  station 
In  one  rapid  motton.  When  the  station  vicinity  on  the  dial  is  reached,  you  then  fine  tune 
the  selector  until  the  station  is  optimally  received.  The  Coarse  Allocation  Program  achieves 
an  Initial,  coarse  deployment  based  primarily  on  range  considerations.  This  coarse  deploy¬ 
ment  is  then  used  in  conjunction  with  the  Fine  Allocation  Program  to  achieve  a  new  final 
deployment  based  primarily  on  visibility  considerations.  The  sequential  operation  of  the  two 
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programs  thus  provides  a  final  deployment  that  has  both  minimized  the  range  from  any  attacker 
to  a  systems  radar  and  maximized  the  combined  radar  system  visibility. 

It  now  becomes  necessary  to  define  a  measure  of  the  "goodness"  of  the  coverage 
or  visibility  of  the  sensor  for  the  points  within  its  defined  sector.  Visibility  was  previously 
defined  as  a  measure  of  the  ability  of  a  sensor  or  sensors  to  detect  a  target  (or  targets) 
wtthin  the  air  space  over  a  given'  terrain  area.  This  measure  can  be  represented  by  a 
range  of  numerical  values  from  zero  to  one  and  wilt  be  colled  the  Visibility  Value.  A 
value  of  zero  will  be  defined  as  there  being  no  visibility  over  a  given  grid  point  for  a 
specified  altitude  range.  For  example,  if  a  grid  point  is  not  within  the  sector  or  range 
of  a  certain  sensor,  a  zero  is  given  to  the  Visibility  Value  for  that  sensor  for  the  grid 
point .  Another  example  of  zero  Visibility  Value  would  be  if  the  MAVD  value  for  a  grid 
point  (for  a  given  sensor)  was  10,000  feet  and  the  probability  of  an  attack  at  that  altitude 
or  above  was  zero.  We  then  would  assign  a  zero  to  the  Visibility  Value.  A  valui  of 
one  would  require  that,  for  the  grid  point,  there  exists  visibility  for  all  possible  altitudes 
of  attack. 

The  method  used  to  convert  MAVD  values  to  visibility  values  is  simple.  First 
a  limit  is  set  on  the  altitude  values  of  interest.  Since  the  emphasis  for  this  paper  is  on 

v1 

low  altitude  attacks  and  since  an  attack  at  high  altitudes  is  visible  to  almost  any  sensor 
allocation,  it  is  unnecessary  to  consider  any  altitudes  above  a  specified  ALT MAX. 

ALT  MAX  will  be  assigned  a  value  for  which  there  is  (a)  essentially  zero  probability 
of  attack  at  altitudes  -  AIT  MAX  or  (b)  considering  terrain  altitudes  and  Irregularities, 
there  is  an  almost  certain  probability  of  detection  of  any  targets  above  ALT  MAX. 

The  Visibility  Value  would  then  be  calculated  as: 


302 


w.  v/,  ALT  MAX  -  MAVD 

Visibility  Value  =  A I  T’JiA’AV - 


where  If  the  MAVD  exceeds  ALT  MAX  we  assign  a  Visibility  Value  of  zero  i.e.,  we  do 
not  allow  negative  Visibility  Values,  Thus  the  Visibility  Value  Is  proportional  to  the  per 
cent  of  air  space  that  the  sensor  can  see  between  the  point  on  the  suiface  of  the  local 
terrain  and  ALT  MAX . 

The  Visibility  Values  as  computed  are  then  written  on  the  computer  drum  In  the 
order  shown  below: 


Visibility  Values  on  Drum 


B  Grid  \ 

Point  N 

1 

2 

3. 

4 

5 

1,1 

.000 

.900 

.905 

1.000 

.500 

1,2 

.100 

.000 

.810 

.150 

.400 

1,3 

• 

.000 

e 

.800 

• 

.000 

• 

.300 

e 

.200 

• 

e 

50,50 

e 

.900 

e 

.900 

• 

.000 

e 

.600 

e 

.750 

240 

,000 

.200 

.800 


1.000 


Thus  each  column  represents  the  Visibility  Values  for  a  possible  sensor  location 
for  all  points  on  the  grid.  Our  objective  is  then,  to  combine  a  specified  number  of  the 
above  possible  sensors  so  thot  the  resulting  "sum"  of  their  coverage  Is  maximum. 

As  pervtously  mentioned  a  now  heuristic  programming  method  called  Complementary 


Programming  was  derived  for  accomplishing  this.  The  method  is  based  upon  the  basic 
principle  of  the  union  of  sets  from  set  theory,  where  each  of  the  columns  in  the  above 
table  of  Visibility  Values  is  considered  an  ordered  set.  The  development,  justification 
and  complete  computer  programs  for  accomplishment  can  be  found  in  references  9  and 
10.  The  present  computer  program  will  handle  a  situation  with  199x  199  grid  size  over 
the  terrain,  38,601  possible  candidate  radar  locations  and/or. 463, 212  possible  candidate 
location  -  pointing  angle  combinations  (if  radar  has  less  than  360°  azmith  capability). 


ATTACK  ROUTE  MODEL 

Having  determined  the  optimal  deployment  of  the  defense,  the  next  step  is  to  turn 
our  attention  to  the  offense.  As  stated  earlier,  the  problem  of  the  offense  may  be  stated 
as,  "given  an  airspace  over  a  specific  piece  of  terrain  that  Is  defended  by  a  ground  based 
air  defense  system, |  find  the  least  risk  route  that  may  be  taken  over  this  terrain  to  reach 
an  assigned  target,  j  Based  on  the  mini-max  principle,  It  is  assumed  that  the  air  defense 

i 

system  is  optimally  deployed  over  the  terrain  and  that  the  offense  has  complete  knowledge 

of  both  the  defense  System  deployment  and  capabilities.  The  least  risk  route  solution  would 

;  \ 

then  specify  at  which  polnt(s)  to  enter  the  defended  air  space,  the  path  to  follow  through 
the  air  space  to  the  target,  and  the  probability  of  survival ,  Such  a  computer  model  has 
been  developed  [ill  and  will  now  be  briefly  described, 

A  survey  of  the  literature  showed  that  very  little  hod  been  accomplished  in  the 
area  of  the  systematic,  objective  determination  of  optimal  attack  routes.  Furthermore, 
of  the  few  methods  proposed,  none  was  capable  of  handling  anything  except  very  small 
problems.  It  wo*  therefore,  decided  to  provide  a  relatively  new  approach  rather  than 
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try  to  build  onto  or  refine  an  older  approach.  The  optimal  attack  route  problem  was 
formulated  in  terms  of  a  cl  issical  network  problem.  This  was  a  natural  approach  in  view 
of  the  grid  overlay  used  for  the  terrain  description  in  the  MAVD  and  Defense  Allocation 
models.  With  regards  to  the  network  description,  we  can  state  our  problem  as:  "Determine 
the  least  risk  route  through  the  network,  where  we  may  enter  the  network  at  any  outer 
node  (Tntersectlon  of  grid  lines)  and  travel  on  any  branch  (grid  line  between  nodes)  in 
either  a  forward  or  lateral  direction. 

The  risk  in  traveling  from  one  node  to  another  in  the  network  is  then  expressed 
as  follows: 

4 

R  » f(V,  Ra,  t) 

\ 

where: 

R  -  Risk 

V  ■  Visibility  factor  (i.e.  is  the  target  visible  or  not) 

i 

Rfl  ■  Range  from  target  to  defense  system 

t  *  The  time  In  which  the  target  is  visible  to  the  defense  system. 

Consequently,  a  least  risk  path  would  in  general  minimize  the  time  the  target 
is  visible,  minimize  the  number  of  times  the  target  Is  visible,  and  maximize  the  range 
to  the  defense  system  (for  the  times  in  which  the  target  is  visible).  Under  this  description, 
each  node  of  the  network  may  be  assigned  a  value  of  risk.  The  "cost"  of  going  from 
one  node  to  the  other  is  then  the  difference  in  risk  from  one  node  to  the  next,  or  the 
probability  of  survival  from  one  node  to  the  next  . 

i 

Under  the  network  formulation,  several  methods  for  solving  the  classical  "shortest 
route  through  a  network"  are  available.  The  most  efficient  methods  are  lineor  programming 
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and  dynamic  programming.  While  either  of  these  methods  can  solve  a  small  problem,  it 

was  found  necessary  to  utilize  dynamic  programming  for  the  laraer  real  world  problem 

because  of  computer  storage  requirements.  For  example,  the  solution  of  a  problem  by 

2 

linear  programming  (Hungarian  algorithm)  would  require  storage  of  at  least  N  points 
(where  N  is  the  number  of  grid  points).  The  storage  requirements  of  dynamic  programming 
are  more  on  the  order  of  4N.  Since  determination  of  a  flight  path  is  a  multi-stage  de- 
cision  process,  dynamic  programming  was  particularly  well  suited  to  the  problem. 

The  method  of  dynamic  programming  is  discussed  in  detail  in  the  literature 

[12]  and  thus  will  be  touched  on  only  briefly.  Generally  speaking,  dynamic  program- 

« 

ming  is  a  method  of  solving  multi-stage  decision  decision  problems,  Unlike  linear 

i 

programming,  there  is  no  standard  mathematical  formulation  of  the  problem.  It  is  a 
general  approach  and  the  particular  equations  used  must  be  developed  to  fit  each  separate 
■case  at  hand.  ! 

As  stated,  we  use  the  same  grid  overlay  as  used  in  the  MAVD  and  Allocation 
Models.  But  under  our  dynamic  programming  formulation  we  let  each  row  (i.e.  nodes 
l/J  with  i  constant)  represent  a  decision  stage  (see  Figure  4),  Each  stage  in  turn,  has 

'  ,  1 

a  number  of  states  associated  with  it.  In  our  case,  the  states  of  each  stage  ore  simply 
the  nodes  of  each  row.  In  general,,  the  states  are  simply  the  various  possible  positions 
In  which  the  aircraft  might  be  at  any  stage  of  the  problem.  In  a  multi-stage  problem 
with  discrete  stages,  (as  in  this  problem)  decisions  are  to  be  made  at  the  beginning  of 
.the  stages.  The  poJicy  decision  to  be  made  at  each  stage  is  the  destination  for  the  next 
stage  i.e.  which  state  in  the  next  stage.  If  is  dependent  upon  the  situation  at  the  time 
of  decision,  upon  the  decision  itself  and  upon  the  stage  of  the  system.  Each  decision 


/ 

/ 


306 


affects  not  only  the  next  stage  but  all  subsequent  stages.  The  solution  of  the  problem  is 
a  sequence  of  decisions  that  yields  the  ieasr  risk  route.  This  is  esseniiuiiy  Beiimun’s 
principle  of  optimality,  "An  optimal  policy  has  the  property  that  whatever  the  initial 
State  and  initial  decisions  are,  the  remaining  decisions  must  constitute  an  optimal  policy 
with  regard  to  the  state  resulting  from  the  first  decision." 

The  particular  version  of  dynamic  programming  used  in  this  problem  computes 
the  flight  path  in  a  "backward"  manner.  That  is,  one  first  starts  at  the  target  and  then 
determines  the  optimal  paths  from  each  state  in  the  previous  stige  to  the  target.  Once 
this  is  done,  the  optimal  paths  from  each  state  in  the  M-2  stage  to  the  M-1  stage  is 
computed.  At  each  stage  only  the  values  of  optimal  paths  need  to  be  stored.  This  pro- 
cedure  is  repeated  until  we  are  at  the  initial  stage  (i.e.  row  one).  At  this  point  all 
of  the  optimal  paths  from  any  of  the  entry  points  to  the  target  are  available. 

As  with  the  allocation  model  discussed  in  the  previous  section  a  two  phase  sequence 
is  used.  The  first  phase  or  calculation  of  the  course  attack  route  is  primarily  predicated 
on  minimizing  visibility  (or  risk)  and  the  second  phase  or  calculation  of  the  fine  course 
route  Is  primarily  to  minimize  e.kposure  time  .  The  solution  procedure  requires  data 
In  the  form  of  two  matrices.  These  matrices  are  (a)  visibility  matrix  and  (b)  missile  flight 
time  matrix.  . 

The  visibility  matrix  provides  the  probability  of  detection  for  each  node  of  the 
terrain  sector,  The  aircraft  altitude,  h,  and  the  MAVD  values  for  each  node  ore  compared. 
If  the  MAVD  value  is  greater  than  h,  the  aircraft  is  invisible  and  the  risk  is  zero.  It 
should  be  noted  that  the  MAVD  value  to  be  compared  with  the  aircraft  altitude  is  always 
the  minimum  value  of  all  the  radar  sites  within  range  of  the  node.  If  the  MAVD  value 


if  less  than  h,  then  the  visibility  value  for  that  node  has  some  probability  value  osso- 

•  J  ...  !sL  !s  T  #■  I  »«•  .<*»  *V>  1 1  ii  irf  n  I  inA  c!nk(  r\r  n  wrtf  infill 

Range  from  the  target,  reflective  surface  of  the  target,  transmitter  power,  size  of  the 
antenna,  etc.  are  all  variables  which  may  affect  the  visibility  of  a  target  to  a  radar. 

A  review  of  the  variables  affecting  visibility  indicates  that  the  effect  of  each  is  dependent 
'to  a  large  degree,  upon  the  range  of  the  aircraft  from  the  radar.  For  this  reason/ the 
range  of  the  target  from  the  radar  having  the  best  MAVD  value  was  selected  as  the  best 
single  variable  to  measure  visibility.  The  relationship  of  range  to  probability  of  detection 
can  be  expressed  graphically  and  is  obtained  from  an  analysis  of  the  perfoimance  speci¬ 
fications  of  the  missile  system  under  consideration.  The  range  value  is  thus  converted 
to  a  visibility  probability  value  based  on  each  missile  systems  specifications.  The  risk 
is  set  equal  to  the  probability  value  for  the  specified  range.  The  detailed  development 

cf  the  model  with  the  procedure  coded  in  FORTRAN  V  language  is  given  in  reference  11. 

( 


SUMMARY 

■Tt  " 

This  paper  has  proposed  a  new  and  unique  approach  for  conducting  comparative 
experiments  or  evaluations  between  air  defense  weapon  systems.  The  sub-models  which 
were  briefly  discussed  were  developed  as  me  of  improving  existing  digital,  computer 
stimulation  experiments.  It  is  believed,  however,  thot  these  submodels  and  developed 
methodok  jies  can  be  utilized  as  the  basis  for  a  completely  independent,  "unified  air 
defense  system  comparison  model."  Such  a  model  could  be  used  for  realistically  analyzing 
and  evaluating  air  defense  systems  in  what  wo  believe  would  be  a  far  more  economical 
ond  accurate  manner  than  is  presently  available  from  simulation  models. 
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FIGURE  2:  MAVD 
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PROBABILISTIC  MANPOWER  PLANNING  FOR  THE 
RESEARCH  AND  DEVELOPMENT  ORGANIZATION 

Larry  H.  Johnson  t 

Redstone  Arsenal,  Alabama 


mriiopucTijii 

Tills  paper  proposes  a  statistical  approach  to  one  of  Hie  oroble-as 
confronting  all  Army  research,  development  and  testing  organizations. 

That  is,  manpower  planning. 

Thougi  the  ■iroblo;'.  addressed  involves  manpower,  it  should  b-  noted 
that  the  Mathematical  techniques  are  a  'plicable  to  all  types  of  inventory 
by  redefining  tlie  paramators  involved. 

The  question  for  long-range  planning  is  not  •.:iiat  should  be  done 
tomorrow,  but  rather  what  can  be  done  today  to  cope  best  with  the 
uncertain  tomorrow.  Management  must  understand  the  alternatives  available 
to  them,  the  risk  associated  with  each,  and  choose  rationally  among  the 
alternatives  rather  than  plunge  into  uncertainty  only  on  the  basis  of 
-  intuition  or  previous  experience. 


DaFPH'i’iqi  OF  PROBLEM 

If  a  given  organization  has  a  large  number  of  programs  planned  for 
the  future,  it  la  usually  reasonable  to  assume  that  the  manpo  ;cr  requirement 
is  somewhat  normally  distributed.  Ho-.rever,  most  .1  &"D  organizations  do 
not  have  a  large  number  of  outstanding  programs  and,  therefore,  the  gain 
or  loss  of  a  single  program  can  have  gross  affects  on  the  required, 
manpower.  This  problem  requires  that  the  “exact",  probability  distributism 
be  known  and  solutions  for  this  problem  are  not  available  in  the 
literature.  >  '  ' 

The  current  need  for  management  planning  techniques  t/ith  relatively 
few  outstanding  programs  no tivatod  the  study  described  heroin. 


A1SUIIPTI0IE 

Tils  study  makes  four  basic  assumptions* 

1.  First  it  io  assumed  that  tho  organization  ,/ill  not  be  required 
to  perform  every  program  for  which  current  planning  exists  and  that  a 
subjoctivo  probability  can  be  associated  with  tho  gain  or  loss  of  each 
program. 


This  article  has  been  reproduced  photographically  from  the  author’s 
manuscript. 
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esse.-.. cl  -hat  each  projrav  is  statistically 
•  •  •  v- •  j.zj ,  ,  c  wi'>  vm on  v’»*  c. u »c c  -—on  v*  one  o^. 

ilj-  no-  c-  of  :-v  other  program. 

3.  .l.Trd,  it  is  r.-J-  r.-d  the. ,  ..orsonnel  of  co.v_.:oa  disciplines  nro 

re  as  j.  .ably  in.e:v  uv.jeabl -.  for  e.-camplo,  any  mechanical  en^;i..ccr  or 
e'.:e  reu’.e  eh.:  ■.elm  could  be  u.ilisocl  on  any  program  whore  cue':  eld'll 
ilj  ; y  ~  t 

4.  Tour'll’.,  each  .-ro.van  b:i:ip  considered  has  a  proposed 

i.’.i :.ia  ,1  or.  date,  and  if  this  due-  is  uncertain,  it  is  assumed  diet  r. 
cud:'. :•! o.vah  probability  for  stertin”  on  various  dawes  can  be  estimated. 


f  Tires,  for  each  cuv.ceandi.v;  prspren,  there  must  bo  a  propre. :  plan 
vhich  delineates  bach  wash  to  bo  performed,  the  time  phasin'',  and  tho 
c-»Vu..r;ce  in  -.:hich  tanhs  .-.ust  be  conducted.  Troa  this  plan,  tho 
:rtani.;aticn  generates  a  time  adjusted  manpower  array  for  pcrfor.-iinj 
all  tasha  under  the  develop:  mu-  plan. 

<  V  .  <>«l,  -  •  hi  ^»-i  .*  a  *.  .  ■  . 

In  offer t  to  promote  appreciation  of  tho  problem  at  hand,  rn 
exc.  .plo  problem  vri.ll  bo  utilised. 

■  Tron  the  development  plan,’  a  time  adjusted  ;  anp over  auvay  for 
perforin^  all  wash's  has  boon  jj&norat>id  and  is  pros  sawed'  in  Ti^ure  1, 

It  is  assumed  certain  that  this  project  vriH  be  conducted,  but  tho  start 
date  is  uncertain.  Utilising  the  wheory  of  ii<pected:  Values: 


:Cv)  (.Jf ) (?,/„  ) (X  i/A  )  (?;  )  (?../*•  )  ) 

.  * 
v;horo  j  *  >L/i 

and  v;hen  ''re'0 projects  are  considered: 

n  k 


C )CCj  ) 


lift*)  *2 2 (?•)(?,)&.•) 

<77  i  *  1  > 


where:  ICf  *■  the  units  of  manpower  required  durin"  period  t 
?^  ■  capture  probability 

?js  probability  that  project  "i”  starts  in  period  "t“ 

X:  «  number  of  :uu-..)0.;er  units,  required  for  project  "i"  durir.f 
the  period’t" 

«*: 

'fr.is  process  ie  i Hue trated  graphically  in  Tipuros  2,  3  and  U, 
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Figure  1  -  Time  Adjusted  Manpower  Array  and  Probable  Start  Date 


Ul 
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Figure  3  -  Actual  and  Expected  Manpower  Requirements  for  February  Start  Date 
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At  this  point  the  problem  solution  sccns  quite  simple;  ho  .-ever, 
oc-c ;-.use  .re  used  expected  values,  the  "risk;'  associated  with  t..c 
solution  cannot  be  evaluated  unless  there  is  a  large  number  o* 
outstanding  projects  (which  usually  is  not  the  case  lor  an  AID 
Organisation),  ./e  mist  therefore  develop  a  model  which  utilises  t..c 
e;v>.c  .  statistical  distribution.  For  '/.is  situation  an  enumeration 
process  has  been  developed  and  is  illustrated  in  Figure  £>.  -Very  possible 
•..orlioad  is  identified  and  the  probability  of  occui-rer.ee  for  each 
is  evaluated.  Considering  that  the  go-ahead  date  for  the  programs 
nay  not  be  "fixed*  but  rather  can  be  expressed  as  a  probabilistic 
function,  wo  expand  the  enumeration  procedure  as  illustrated  in  Figure  C 
where  the  additional  uncertainty  is  accounted  for  by  t, ,  t*,  cue. 

A  technique  ha3  thus  been  developed  for  enumerating  the  total  array 
of  possible  workloads  for  an  organization  and  the  probability  associated 
with  each.  Thin  concept  can  readily  be  adapted  to  fit  any  particular 
problem  that  one  nay  have. 

uxa-i  cost  resnop 

Given  the  total  array  of,  possible  manpower  requirements  developed 
above,  the  corporation  is  now  faced  with  the  problem  of  determining  -the 
cost  economical  method  of  performing  any  given  workload.  That  is,  if 
management  wore  to  assume  that  the/  knew  specifically  which  ono  of  the 
workloads  Trill  occur,  how  can  they  most  economically  perform  the  task 
realizing  that  if  the  workload  exceeds  capacity,  they  nay  choose  to: 

1.  hire  additional  employees 

2.  work  on  an  overtime  basis  or 

3.  subcontract  a  portion  of  the  work. 

If  capacity  exceeds  the  workload,  management  may -choose  to: 

1 .  continue  on  an  over-staffed  basis  or 

2.  lay-off  some  employees 

Management  must  recognize  that  with  each  alternative,  the  total 
co3t  for  conducting  the  workload  -.rill  vary.  For  example,  now  employees 
roust  be  trained,  overtime  costs  premium  -.rage,  employee  efficiency 
decreases  with  overtime  and  subcontract  personnel  may  not  be  as 
affective  as  regular  eaployoos. 

Total  capacity  for  the  organization  during  any  period  is 
therefore  given  by: 

■uhore; 

is  the  efficiency  factor  for  each  type  of  manpower  ioverti:*, 
subcontract,  etc.) 
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DURING  PERIOD  t 


~ X  is  ~-..q  of  fee Live  capacity  of  manpower  available  duriwg  period 

Zt'jt  is  tic  number  or  experienced  employees  available  during  period  o. 
i3  the  number  of  now  employees  available  during  period  t. 

XJ  jt  i3  the  number  of  overtime  unit3  which  can  bo  ‘.jerked  by 
experienced  onnloyoe3  during  poriod  t» 

^s,£  is  the  number  of  overtime  units  which  new  enplpyooo  can  work 
during  period  t. 

X-i't  is  the  number  of  subcontract  personnel  available  during  period  t. 

Rote  that  X3>r  which  is  the  number  of  full  time  employees  to  be 
terminated  during  period  t,  is  not  included  in  the  equation*  Unit 
costs  for  cacli  type  of  manpower  must  also  be  availablo. 

The  problem  is  one  of  determining  the  optimum  manpower  schedule 
for  a  given  workload  which  permits  the  organization  to  operate  for  the 
duration  of  the  planning  period  uith  nrfrvimnn  labor  costs* 

Requirements  and  data  inputs  for  the  minimization  problem  ore  ideal 
for  solution  by  the  Simplex  Linear  Programing  Technique  where  the 
constraints,  due  to  management  policies,  labor  agreements,  etc.,  limit 
lie  range  of  values  for  X*  *  The  Simplex  not  only  provides  a  manpower 
plan  for  each  workload  but  also  the  total  cost  for  each  plan. 

Mote  that  in  seme  cases  the  Simplex  may  indicate  to  hire  employees 
one  month  and  terminate  them  the  next.  This  is  not  fault  with 
the  mathematics  but  rather  fault  with  management  policies.  The  I-Snimum 
Cost  Technique  is  therefore  a  good  indicator  for  restraining  management 
labor  policies. 

risk  :lthod 


In  the  previous  discussion  it  may  be  noted  that  the  probability 
associated  with  each  workload  was  ignored.  How  then  can  the  element  of 
risk  be  considered  in  the  management  plan? 

Given  the  probabilistic  manpower  requirements,  management  needs 
a  decision-making  policy  which  allows  thorn  to  plan  for  a  theoretical 
worltLoad  and  adjust  with  minimum  consequence  to  the  actual  workload 
".•'hen  it  occurs.  The  Idnimum  Risk  Method  will  provide  such  a  plan. 
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Ccns luering  all  of  the  possible  workloads,  tic  so  Lion  of  r.:k: 
introduced,  ‘.rhcrc  rial:  for  a  giver,  workload  is  defined  as  lie  core  of 
noising  a  transition,  in  Inner  intervals,  from  none  planned  ;ia.’..»r  rr 
level  to  the  appropriate  liininun  Cose  Plan.  In  short,  rich  is  the 
cost  of  adjusting  to  the  workload  lint  actually  occurs.  Once  the 
workload  becomes  known,  an  adjustment  or  transition  is  made  and  the 
appropriate  Kinimun  C  ost  schedule  is  followed. 


The  problem  no:;  is  to  identify  a  manpower  planning  level  which 
minimizes  total  risk  for  the  tnumeratid  range  of  worltLoads . 

Letting  represent  the  minimum  cost  for  the  k  workload  and 
C\  represent  tiie  cost  of  adjusting  from  the  planned  level  to  the 
appropriate  ilininum  Cost  Plan  and  completing  the  job,  then  the  risk 
II  is  given  by  .  „ 

Vck-ck 

The  expected  risk  (s£)  for  each  of  the  k  workloads  is  given  by 

\  -  <£> 

’.here  P^  is  the  probability  of  occurrence  determined  by  enumeration. 

The  total  risk  R(  • )  for  any  manpower  plan  is  therefore  given  by 

n(-)^  ‘  2  Pk(o^  -  cp 

Our  problem  is  now  to  identify  a  manpower  plan  which  minimises 
X(*).  A  dynamic  programming  tcciinique  for  minimizing  R(*)  has  not 
been  developed;  however,  we  can  iterate  a  solution  as  illustrated 
in  tiie  following  problem.  • 

Example  Problem 

Suppose  the  scheduling  period  is  U  months  and  the  initial  number  of 
experienced  employees  i3  60.  The  organization  has  tiro  outstanding  project 
with  estimated  capture  probabilities  of  0.6  and  0.3  respectively  and  it 
trill  not  be  known  until  1  January  if  the  projects  will  be  funded.  Tiie 
t;pe  A  manpower  requirements  for  each  project  are  presented  in  Table  1 . 

The  projected  manpower  requirement  t/ithout  consideration  of  tiie  two  new 
contracts  i3  also  3hown. 
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. i  i  v.  » t  i  1  .  *W  a  4. 1  Ol i  Ol 

. , . ; ; ■  >uwe r  Re  eu ;  rein  c  r.ts 

Probability 

January 

February 

I 

March  1  Ap.nl  ; 

Workload  without  new 

c  w  *  1 1>  *  *1  c  i>  s 

1.0 

50 

70 

00 

| 

GO 

Contract  I 

O.G 

10 

10 

20 

30 

I 

;  Contract  II 

0.3 

30 

20 

20 

10 

ooewanv  solidus  and  labor  a^rccsients,  com  .rain«a  - 


sucn 


100 

0^X2t=  20 

,s\,a  20 

03  30 

0  sx  Ss  30 

Oft 

03  X6.t  3  20 


.  1.  and  efficiency  factors  for  the  various  types  of  Manpower  are 

resumed  in  'fable  2.  It  i3  also  assunod  that  no;/  employees  can  bo 
, rained  ■.ritiiin  one  tino  period. 


326 


Table  2.  Unit  Coats  and  Efficiency  Data  for  Ilininum  Risk  problem 


Unit  Costs  ($) 

Identification 

Efficiency  Factors 

Cj=  100 

Experienced  employees 

A,  =  1.  OU 

C,=  130 

New  employees 

X2  -  0.  50 

O 

W 

II 

tn 

o 

Mandatory  terminations 

C4  =  150 

Experienced  employee  overtime 

\4  =  0.70 

Cs=  150 

New  employee  overtime 

Xs=  0.35 

C,=  170 

Subcontracts 

\  ~  0.  80 

Hie  .jroblen  is  to  dvtemLne  theoretical  values  for  ^  1  ,.L,  . ,  e.c., 
which  minimise  the  total  risk  for  this  planning  situation.’  ’ 

deferring  to  the  caaplctc  enumeration  technique,  there  arc  four 
possible  trorkloads  with  probabilities  of  occurrence  ar  calculated  and 
sho  rn  in  Table  3«  The  ;  inirrun  Cent  plan  for  each  or  the  four  possible 
•..’orlclor.h;,  f.etor.ai.i.  d  V>,  the  dirnl.e:  Technique,  is  presented  in  fable  J;. 

Since  the  primary  interest  for  planning  is  full  tine  enployeos,  and 
the  Minimum  Coat  plans  of  Table  U  indicate  that  portions  of  the  workload 
should  be  subcontracted,  tho  workload  and  Minimum  Cost  arrays  aro  modified 
to  rofloct  only  the  in-house  efforts.  Tho  in-houao  offort3  are  determined 
hy  subtracting  the  subcontracts  from  Tables  3  and  h,  and  results  of  this 
operation  arc  presented  in  Tables  5  and  6, 


Table  3,  Enumerated  ./orkload  Requirements  for  liinimum  Risk  Problem 


Possible 

Probability 

Manpower  Requirements 

Workloads 

of  Occurrence 

January 

March 

W  j  *  M*  M* 

P*  p,  »  0.28 

50 

70 

90 

60 

W,  =  M ,  Mj 

P,P  ?=  0.42 

f>0 

80 

110 

90 

w3  =  m’J  m2 

p,  p2=  0. 12 

80 

90 

110 

70 

W4  =  M 1  M; 

Pi  Pj  ~  0. 18 

90 

100 

130 

100 
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T«hla  6.  Minima  Coat  Plans  for  3h-House  Workloads 


Minimum 
Cost  Plan 
for 

■ 

January 

February 

March 

April 

Total 

Cost 

($) 

0 

B 

2m 

X)  2»  60 
X2>  20 

X  ij  =  80 

X,.=  60 
X,>  20 

29,600 

m  . 

KB 

0.42 

x  1.1=  60 
XM-  10 

Xtl2=  70 
X2,2=  20 

gQQjl 

B 

35, 757 

Wi 

B 

X1(1-60 
X2l,=  20  . 

Xll2“  80 

x2,2  =  10 

Xu-  90 
X4.J=  5.71 

35, 757 

t 

0. 18 

Xul-  60 
Xj,  1“  20 
X4.!-  5.71 

Xi,2-  80 
Xj,j  “  20 

XU-  100 

43, 057 

The  next  procedure  Is  to  Iterate  costs  and  risks  fo:  all  feasible 
values  of  X (  /  and  Xi>(  so  that  the  minimum  R(  • )  can  be  i  ->tlfied.  It 
asy  be  noted  in  Table  6  that  X/w  is  60  for  all  four  wor*  ‘ 's;  therefore, 

there  is  only  one  feasible  solution  for  X  tft  .  However.  varies  from 
0  to  20.  The  problem  then  is  to  determine  a  value  (xj! ;  mich  minimizes 
R(0  keeping  in  mind  that  the  objective  is  to  adjust  to  u.  -  lUnlmum  Cost 
plan  in  the  most  expedient  and  economic  manner  consistent  with  the 
manpower  constraints. 

The  results  of  the  iteration  process  for  each  of  ( the  fox  possible 
workloads  is  presented  in  Tables  7  througi.10  -.here  X2.I  uas  varied  from  0 
to  20  in  increments  of  f>  units.  Increments  of  five  units  each  ./ere 
arbitrarily  selected  for  simplification  of  calculations  in  this 
illustrative  problem,  iiote  the  heavy  line  in  each  of  the  tables.  Tills 
line  indicates  when  the  level  of  full  time  enploynent  reaches  the  IHniaum 
Cost  plan  for  that  particular  workload.  Toted  cost  for  each  of  the  trial 
solutions  is  also  presented  in  Tables  7  through  10. 

Summary  of  the  expected  risk  calculations  with  the  co3t  data 
from  Tables  7  through  10  is  presented  in  Table  11.  The  term  Rt*  ^  nay 
then  be  calculated  by  the  equation  R(  • )  *  ?  Hu  for  each  of  the  xX, 
solutions.  The  R(‘)  data  are  tabulated  in  Table  12  where  it  is  sbwon 
that  R(* )  is  minimum  when  X*  ,  is  10  units. 
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Table  9.  Enumerate  cl  Risk  i=lan  if  Con  -met  JJ  is  .teceived 


XM  = 

6Q 

x4,i  = 

14.3 

Xi.,- 

60 

XI,J3 

5 

XM  = 

10.7 

XUS 

60 

X,ti- 

10 

X4,!  = 

7.1 

Xl,l» 

60 

XM* 

15 

X4.  1  ~ 

3.6 

Table  1,..  Sn-jfloratcd  2isi:  x*lan  if  Contracts  I  and  II  are  Iteceivcd 


Hi: 


Solution 


3G.980 
36.3G0 

35,757 
36, 255 
36, 755 


29. GOO 

1 

29,600 

2 

35.757 

35.757 

35,757 

35,757 

35,757 


38, 455 
37,685 
36,917 
36,335 

35,757 
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.«iat  does  this  solution  moan?  The  Iiininun  Sisk  plan  for  this 
illustrative  problem  is  to  retain  the  60  experienced  employees  who 
will  be  available  for  January  and,  in  .addition,  hire  10  new  employees 
before  January  so  that  their  services  will  be  available  during  the 
first  month.  If  workload  I,  occurs,  the  corporate  plan  will  be  as 
shown  in  Table  13.  If  rorkloads  lJt  or  .<4  occur,  the  corporate 
plan  will  be  as  shown  in  Tables  lit  through  16  respectively* 


Table  13.  Minimum  Risk  Plan  Jilhout  Hew-  Contracts 


KW 

January 

February 

March 

KIH 

60 

10 

BB 

Xll2  =  70 

P 

Xli3=  80 

XM-  60 

HO 

x2i!  =  10 

X(j(3  *  12.5 

*3,4*20 

Table  11».  Minimum  Risk  Plan  if  Contract  I  is  Recoivcd 


X'.« 

January 

February 

March 

April 

60 

10 

Xi,i*  60 

XJi2  *  70 

X,,,-90 

XM»  90 

X*,j-  10 

*3,3=20 

*4,3  =  5.7 

*6,3  =  20 

Table  15.  IHniatum  Risk  Plan  if  Contract  IX  is  Received 


m 

Xj.i 

January 

February 

March 

April 

60 

10 

*i,i3  GO 

Xll2=  70 

Xli3=  90 

X1>4=  70 

Xj,i=  10 

X2>2=  20 

*4,3  =  5.7 

XJl4  -  20 

*4.1  =  7.1 

*4,3  =  7.1 

*8,3  0  20 

X{fl=  12.5 

Xgt2  —  6.2  5 

* 
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Table  16.  Fdnircum  Risk  klan  if  Contracts  I  and  II  ar 


coivcd 


x;,, 

January 

February 

March 

April 

60 

10 

X|,,-  60 

X1>2=  70 

X,,3=  90 

X  i,4  =  100 

x2tl=  10 

X2,2-  20 

X2,3  =  10 

X4ti=  12.9 

*4.2  =  14.3 

X4,J=  27.  1 

XM-  20 

x6|2  =  12.  r. 

XEl3=  20 

The  organization  is  hereby  presented  vrith  a  strategy  for  planning 
the  future  manpower  requirements  in  the  face  of  unccrtaintp.  A 

thenatical  simulation  has  been  developed  which  can  assist  management 
understanding  the  problem  and  the  effects  of  various  plans  available 
them.  It  is  felt  that  this  approacn  can  be  c  o:y uteris  ed  and  provide 
management  with  a  rapid  assessment  of  t.ic  situation  at  an;-  given  time. 


* 


\ 

I; 

I! 

1  - 
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ANALYSIS  OF  FACTORIAL  EXPERIMENTS  IN  NON-CONNECTED  BLOCK  DESIGNS* 


B.  M.  Ku  kjlan  and  R.  C.  Woodall 
Harr  Diamond  Laboratories 
Washington,  D.  C. 

1,  Introduction 

Hie  object  of  the  analysis  is  to  estimate  tht-  effect  of  v  treatments  cr.  the 
response  of  a  device  in  the  presence  cf  b  extraneous  side  effects  (blocks),  whose 
effects  are  removed  from.  the  treatment  effects.  7r.e  treatments  may  be  simple 
treatments  involving  only  cno  factor,  or  they  may  be  treatment-ccrabinaticns 
involving  several  factors  applied  simultaneously  tc  the  device.  In  the  latter 
case,  the  effect  of  each  factor  and  the  effects  of  interaction  between  factors  are 
also  estimated.  Tee  results  presented  here  require  no  restrictions  on  the 
experimental  design.  That  is,  the  design  may  be  unbalanced,  treatments  may  be 
missing,  there  may  be  an  'unequal  number  of  observations  per  cell,  etc. 

2.  Model 

The  model  is  the  fixed  effects  model: 
yijk  =  p  +  ti  +  bj  +  Eijk 


where  p  -  overall  constant 

t,-  1th  treatment  effect,  i  ■  1,  ...»  v 
bj-  Jth  block  effect,  J  ■  1,  b 
yijk“  ^  observatlon  of  treatment  1  in  block  J 

k  *  lj  • • ■ »  iii 


e^j^-  experimental  error  in  assumed  to  be  tl(0,cr) 


3.  Incidence  Matrix 

The  design  of  the  experiment  is  characterized  by  the  incidence  matrix,  M, 
which  is  a  vxb  matrix  whose  elements  n. .  are  one  if  treatment  i  is  applied  in 
block  J*  and  are  zero  otherwise.  For  example,  if  v  ■  3  and  b  *  3,  the 
incidence  matrix  might  be; 

Treat-  Blocks 

ment8  123 

1  110  t^  is  applied  in  blocks  1  and  2 

2  011  tj  la  applied  In  blocks  2  and  3 

3  101  t^  la  applied  in  blocks  1  and  3 

Lst  ijj  be  the  nutber  of  observations  on  treatment  1  in  block  J .  Then 
L  -  (Ujj))  -  (toy 

*This  la  a  condensed  version  of  a  paper  which  Is  to  appear  In  a  national  journal. 
This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 


Preceding  page  blank 
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*4.  Definition  of  Connected  Designs  and  Sets  of  Connected  Blocks 

A  connected  design  Is  one  in  which  all  the  blocks  are  connected  by  a  chain  of 
treatments.  In  the  example  ng.  1  ,  blocks  1  and  2  are  connected  by  treatment  1 
and  blocks  2  and  3  are  connected  by  treatment  2  -  hence  all  blocks  are  connected 
and  the  design  Is  said  to  be  a  connected  design.  The  nunber  of  sets  of  connected 
blocks  Is  one,  and  the  set  consists  of  {bp  b2,  b^h 

In  the  exanple  below,  blocks  1  and  2  are  connected  by  treatment  1,  but  there 
is  no  treatment  which  connects  either  block  1  or  block  2  to  block  3  -  hence  the 
design  Is  said  to  be  non-connected.  There  are  two  sets  of  connected  blocks 
(bp  b2)  and  (b^h 


110 
0  10 
0  0  1 

5.  Review  of  Solution  for  Connected  Designs  with  Mo  Missing  Treatments 

The,  least-squares  solution  obtained  by  minimizing 

v  b  *ij  , 

s  *  l  l  l  nn  <yL1k  -  v  -  t±  -  b,)- 

1-1  j-l-  k-l  1J  Wk  ■  1  J 

with  respect  to  u ,  ,,and  bj ,  and  eliminating  y  and  bj ,  gives  the  reduced  normal 
equations:  C  £  •  Q,  where 


£  -  (£p  t2,  ...  £v) ,  the  vector  of  treatment  effects 
C  -  R  -  L  K”1  l' 

Q  -  T  -  L  K”1  B 

and:  R  is  a  diagonal  matrix  with  diagonal  elements  equal  to  the  number  of 

*  b  i 

observations  on  tp  i.e.  r,  «  l  1J,  1  ■  l,  ....  v 

J«1 

All  >  0  In  the  no  missing  treatments  case 

K  is  a  diagonal  matrix  with  diagonal  ^laments  equal  to  the  nunber  of 
observations  in  block  J,  i.e.  k  *  t  l. ,  >0,  J  ■  1,  . . . ,  b 

j 

L  Is  the  matrix:  L  -  (C^jn^j)) 

T  ■  (Tp  Tg,  ....  Ty)  Is  a  vector  of  treatment  totals,  i.e. 

T1  ■  ?  F  rWn*.  1  -  1,  .....  v 
,  J-l  k-l  10 

B  -  (Bp  Bgt  ....  Bb)  Is  a  vector  of  block  totals,  i.e. 
v  41J 

B,  -  J  J  J  "  1,  •••,  b 

J  1-1  k-l  J  J 


(2) 
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A  +  -f 

The  solution  t  =  C  Q,  where  C  is  the  generalized  inverse  of  C,  gives  the 
rMyrirniinv-varl sner.  unbiased  estimates  of  the  tt ,  i  =  1,  . ..,  v,  subject  to  the 

V  X 

constraint  j  t.  =0.  The  rank  of  the  matrix  C  gives  the  nuriber  of  linearly 
i=l  1 

independent  treatment  effects  which  can  De  estimated,  hence  th.e  degi-ees  of 
freedom  associated  with  treatments.  Thus  for  the  connected  design,  no  missing 
treatments  case,  we  have  the  following: 

Treatment  effects  ^  *  C+  Q 

Variance-covariance  matrix  V(t)  ■  C  o 

Suns  of  squares  due  to  treatments  SS(t)  =  Q  ■  C  £ 

Degrees  of  freedom  d.f.  (t)  *  Rank  C  *  v  -  1 

Let:  w  be  the  total  number  of  observations  in  the  experiment, 

Y,  the  vector  of  observations,  ((n^y^)^  x  ^ 
v  b  ‘ij 

Q  ■  y  J  V  n,  ,y, ,  the  grand  total  of  the  observations . 
i-1  5-1  k-1  1JK 

Then  the  analysis  of  variance  table  is  given  by: 

Source  of  Variation  Suns  of  Squares  Degrees  of  Freedom 

Treatments  ^  '  Q  Rank  C  *  v  -  1 

(adjusted  for  blocks) 

Blocks  (unadjusted)  b'k"*1  b  -  02/W  b  -  1  ,  (3) 

Error  Y/Y-t  /  Q-B/  K“'^  B  w  -  Rank  C  -  b 


Total  Y'  Y  -  02/w  w  -  1 

For  the  factorial  case,  the  notation  developed  by  Kurkjian  and  Zelen  in 
"A  Calculus  for  Factorial  Arrangements",  Annals  of  Math  Stat,  Vol.  33»  No.  2, 
June  1962,  will  be  used.  Two  operators,  «,  symbolic  direct  product  (SDP),  and  x, 
direct  product  (DP)  are  needed. 

The  SDP  is  used  to  order  the  combinations  of  levels  of  the  various  factors, 
illustrated  by  the  following  example:  Assume  three  factors  in  the  experiments, 
two  at  two  levels  and  one  at  three  levels.  Let  6'  *  (1,  2,  . . .  „  m  )  be  a 
vector  designating  the  levels  of  the  scn  factor,  swhich  has  m  levels.  Then: 
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The  final  vector  rives  a  particular  order  of  the  combinations  of  the  levels  of 
the  three  factors,  and  is  obtained  by  setting  the  first  two  factors  at  level  1 
and  running  through  all  levels  of  the  third  factor;  setting  the  second  factor 
at  level  2,  running  tiircugh  all  levels  of  the  third  factor  again;  and  finally 
setting  the  fiitt  factor  at  its  second  level  and  repeating  the  sequence  on  the 
second  ani  third  factors  again.  The  procedure  can  easily  be  generalized  to 
any  number  of  factors  at  various  levels. 


The  DP  is  the  matrix  multiplication  defined  as  follows : 


•'i 

V  X  n  X  Bp  x  q 


That  is,  each  element  of  A  is  multiplied  times  the  entire  matrix  B,  by  ordinary 
multiplication  of  a  scalar  times  a  matrix. 

Now  let  Aj,  A2i  •••»  An  be  n  factors  in  the  experiment 


“llB 

a12B 

...  a^B 

‘aE 

a 

*22* 

a 

a  a  a 

•  • 

a 

”hiB 

• 

{WB 

a  a 

■  a 

a_B 

mn 

at  m,,  m2,  ...,  m^  levels,  respectively. 

The  nunber  of  treatments  (or  treatment-combinations)  resulting  from  applying  all 
the  factors  simultaneously  at  all  combinations  of  their  levels  is  v  ^ 

Let  /.  .  ,  s  be  the  ith  treatment -comb ination  where  factor  A,  is 

ul»  l2»  •*"  V 

at  level  i1>  factor  is  at  level  1^,  etc.,  and  order  the  treatments  by  the  SDP 
of  the  levels  of  the  factors. 

For  example,  if  n  ■  2,  raj  *  2,  m2  ■  3»  then: 
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; 


*  6. 


ll 


rnl 

— 

t,.,  ' 

1  xa  : 

12 

t2! 

t12 

1? 1 

t  i 

t  1 

21 

;  Thus  t  = 

J 

'ij 

l21 

22 

H 

l  t?-2  I 

23 

te 

l>J 

Thus  treatment  1  Is  the  combination  with  both  factors  at  level  1,  treatment  2  Is 
the  combination  with  the  first  factor  at  level  1  and  the  second  factor  at  level  2, 

etc. 


Let  a  (i  )  be  the  main  effect  of  factor  A  at  the  i  level, 


a  rt  4 

rs '  s,  r)  be  the  second-order  interaction  effect  between  factors 

and  A  at  levels  i  and  i  ,  respectively, 

S  i*  3 


4 

a,-  (i.,  i i)  be  the  n“  -order  interaction  effect  between 

id  . . .  n  i  d  n 

the  n  factors  at  levels  i^,  i2,  ...  ,  in,  respectively. 

Then  the  1th  treatment  expressed  in  terns  of  the  main  and  interaction  effects  of 
the  factors  Is: 


t(i1,  i2,  ...  ,  iR) 


k  VV  *  l  l  W  V  +  •••  +  *1,2  ...  n(1l>  l2-  V 

1  £S  < r.<n  iW. 


For  the  previous  example,  we  get  the  relationships: 
-  tn  -  a^l)  +  a2(l)  +  a12(ll) 
t2  -  t12  -  a1(l)  +  aj(2)  +  a12(l 2) 

•  •  • 

•  i  • 

•  •  • 

tg  ■  t23  ■  a1(2)  +  82(3)  +  ^(23) 


/ 


/ 
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Let  represent  a  general  interaction  term  vector  of  effects  where 
X  *  (x^,  Xg,  "...  ,  x^)  such  that 

Xj  ■  1  if  factor  is  present  in  the  interaction  term  and 
Xj  ■  0  if  not. 

n  x. 

The  number  of  elements  in  the  vector  is  1  ,  again  the  elements  are  assured 

to  be  in  the  order  defined  by  the  SDP  of  the  levels  of  the  factors  involved,  and 
the  order  of  the-fnteraction  is  given  by  p  *  J  x. . 

i-1 

The  nunber  of  interaction  terms  for  an  n-factor  experiment  is  211  -  1,  which 
is  the  number  of  combinations  of  zeroes  and  ones  in  the  vector  X,  excluding  all 
zeroes. 

Continuing  with  the  example,  there  are  2°  -  1  ■  3  interaction  terms  as  follows: 


Adding  the  constraints  that  the  sim  of  the  effects  In  an  interaction  tern 
over  all  levels  of  any  one  factor  is  zero,  i.e. 

ms 

£  as(lg)  “  o  S  =  1,  ,  n  (6) 


V1 


in 


■v 

*  a_(i„  ij  * 

~s 

etc.. 


r  =  1,  ...,  n 


4  -  ~  ill  ars(ir’  V  “  0  cj-i  '  r  *  s 

l  .  rsr,  s  i  =  1  s  -  i,  ...,  n 

r=  i 


the  relationships  in  (,t)  can  be  solved  uniquely  for  the  interaction  effects  in 
terms  of  the  treatment  effects,  giving: 


where 


(7) 


X  X 

Mjj  =  1  x  1^2  2  x  . . .  Mnn  (Using  DP  multiplication) 

x, 

and  Mj1  *  -  Jif  if  xA  *  1 


■  1±  «  (1,  1,  •••*  D  i  x  ^  if  xi  *  0 

where  I.  is  the  identity  matrix  of  order  m. ,  and 

1  J  1 

1 s  a  matrix  of  all  ones  of  oWer 

lhus  for  the  example,  the  constraints  are: 

2  3  2.  2  2 

l  a1(i)  -  l  a2(i)  -  £  a^Cil)  «  \  8^(12)  -  l  a^C.13) 

i*l  1«1  1*1  i*l  i*»l 

■  j1a12m)  ■  JLa12<21)  '  0 


And  the  interaction  effects  expressed  in  terms  of  the  treatment  effects  are 
given  by: 


343 


A 

t 


7 

i 


i 

i 


.The  analysis  of  variance  table  is  given  by: 


Source  of  Variation 

Sums  of  Squares 

Degrees  of  Freedom 

*1 

SSfajj  ) 

Rank  TY 
*1 

X 2  (Adjusted  SS(%)  Rank 

for  Blocks) 


4 

4 

• 

• 

• 

X2n-1 

®<Vl} 

?Xgn-l 

Blocks  (Unadjusted) 

b'  k_1  b  -  g2/w 

b  -  1 

Error 

y'y  -  t'Q  -b'  vrl  B 

w  -  Rank  C  -  b 

Total 

Y/  Y  -  G2/w 

w  -  1 

(9) 


Fran  a  computational  viewpoint,  the  calculations  involved  in  (8)  can  be 
greatly  reduced  by  eliminating  the  elements  of  each  a.,  which  are  linearly 
dependent  because  of  the  constraints  in  (6).  The  total  number  of  elements  in 

all  the  vectors  is  ^  +  1)  -  1.  *hlle  only  (v-1)  are  linearly  Independent 

In  the  connected  design  case.  If  all  elements  involving  any  factor  at  its 
highest  level  (each  of  which  can  always  be  expressed  in  terms  of  other  elements 
in  that  term  using  the  relationships  in  (6))  are  eliminated,  there  will  result 
(v-1)  independent  elements. 

Then  let  be  a  vector  containing  the  linearly  independent  elements  of  ^ 
(selected  as  above)  and  let  be  the 
equations  corresponding  to  those  in  (8)  became: 

V  &  V ■?  «i0+« 

Var  <V  -  ^  \  C+  M '  •  o2 

009  (\,  \  C+ 

SS  (ax)  -  SS(ax)  *  Tx 
f x  ■  fx  ■  Rank  *  9^ 


Tx 


m/  1  +  J 


-  l)xi 


(10) 


corresponding  rows  of  M^.  Then  the 
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The  dimensions  of  the  van  and  cov  matrices  are  reduced  from  xi  to 

n  x  — 

^(ny-l)  i,  and  the  matrix  is  non-singular,  so  SS  can  be  computed  using 
the  regular  inverse. 


The  analysis  of  variance  table  (9)  remains  the  same,  since  the  sums  of 
squares  and  degrees  of  freedom  are  equal. 


6.  Non-connected  Deslms  and  !'lssing  Treatment  Solutions 

If  the  design  is  not  connected,  then  additional  constraints  are  needed  to 
find  a  unique  solution  to  the  reduced  normal  equations  C  %  «  Q. 

fU 

Let  z^  be  the  number  of  sets  of  connected  blocks  and  let  be  the  i  set, 
i  ■  1,  2,  ...,  z^  Let  z2  be  the  nunber  of  missing  treatments  (i.e.  the  nunber 
of  r^  =  0).  Then  there  must  be  z^  +  z2  constraints  to  find  a  unique  solution  to 
the  reduced  normal  equations. 


If  the  constraints  are  taken  to  be: 


t1  “  0  ,1  ■  1,  2,  ...»  *1,  and 

tA  *  0  for  each  i  such  that  ■  0, 

that  is ,  if  the  sum  of  the  treatments  associated  with  each  set  of  connected 
blocks  Is  zero,  and  each  treatment  that  is  mssing  is  assumed  to  be  zero,  then 
the  solution  t  ■  C  Q  with  C  and  Q  as  previously  defined,  satisfies  the 
constraints.  The  analysis  of  variance  table  (3)  remains  the  same  except  that 
the  degrees  of  freedom  for  treatments,  Rank  C  *  v  -  z,  -  Zj-  '  1 

Now,  in  the  factorial  case,  the  problem  is  to  find  the  relationships 
resulting  frcm  the  additional  constraints  on  the  t.'s,  select  a  set  of 
(v  -  z.  -z2)  Independent  ix.'s,  and  compute  the  corresponding  suns  of  squares 
and  degrees  of  freedom  forAthe  analysis  of  variance  table. 


To  determine  the  relationships  on  the  a's  in  addition  to  those  in  (6)  let 


1 

1 

I _ 

—  - 

% 

&  » 

• 

• 

M  - 

• 

« 

^Xgn-l 

v-1  x  1 

^n-l 
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That  Is,  a  consists  of  all  the  elements  in  the  ay  vectors,  as  defined  in  (10) 
and  M  consists  of  the  corresponding:  rows  of  each  .‘!y.  Then  the  system  of  equations 
to  calculate  the  (v-1)  <1  elements  is: 

i  =  JIt  =  inc+Q  (ii) 

Linear  relationships  among  the  rows  of  M  C+,  and  hence  among  the  elements  of 

a,  can  be  determined  by  numerical  techniques.  If  the  rows  of]?  C+  are  arranged 
so  that  rows  correspond!.-  -  to  elements  of  main  effects  are  first,  then  those  of 
second-order  interaction  terms,  then  third-order,  etc.  and  a  pivotal-method  is 
used  in  which  rows  are  interchanged  only  when  necessary  tc  remove  a  zero  element 
from  the  diagonal,  elements  of  terms  of  lowest  order  possible  can  be  selected  for 
the  independent  elements,  and  the  remainder  expressed  in  tents  of  those  elements. 

Hie  linear  relationships  so  determined  can  be  used  to  categorize  terms 
involving  the  dependent  elements  as  being  aliased  -with  independent  terms  for  which 
the  coefficients  are  non-zero,  or  unestimable  if  all  coefficients  are  zero. 

Having  so  determined  a  set  of  linearly  independent  elements,  reduce  a  by 
eliminating  the  dependent  elements,  getting 


~  M  C+  Q 


•where  M  contains  the  rows  of  M  corresponding  to  the  elements  in  a,  and  where  the 
Xp  1  «  1,  . ..,  k,  represent  those  interaction  terms  for  which  at  least ‘one  element 

is  among  the  final  set  of  independent  elements.  Some  terms  may  not  appear  at  all 
(if  all  elements  associated  with  that  term,  have  been  eliminated),  while  others  may 

n  x 

have  degrees  of  freedom  less  than  -1)  1  (if  only  part  of  the  elements  have 

been  eliminated).  Then  the  following  relationships  hold; 


Var(^x  ) 


-  M  t  ■ 

y 

■i  \ 


1  ws  X 

7  M  c+  Q 


a =/  O 


2!  K 


Cov(9L  ly  )  »  (T 
V  J  V2 

SS(eu  )  -  \  T"1 
Ai  *1  x1 

fy  ■  Rank  J 


i“l,  .. .,  k 


i  *  J 
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k  A t 

If  the  design  is  orthogonal,  then  \  SS(a~  )  =  t  Q  =  SS( treatments) . 


Ihe  analysis  of  variance  table  i3  given  by: 


Source  of  Variation 

Suns  of  Squares 

Degrees  of  Freedom 

*1 

SS(a^) 

i.  ^ 

II* — * 

*2 

(adjusted 
for  blocks) 

ss(ax2) 

Rank  \ 

X2 

• 

• 

• 

*k 

• 

• 

SS(a^) 

• 

• 

Rank  T 

*k 

Blocks  (unadjusted) 

BK'fe  -  a2/w 

b-1 

Error 

y'y  -  t  Q  -  B  KiB 

w  -  Rank  C  -  b 

Total 

y'y  -  a2/w 

w  -  1 
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DESIGN  OF  FIELD  TEST  PROGRAMS  AND  STATISTICAL  TECHNIQUES 
FOR  ANALYSIS  OF  THE  PERFORMANCE  OF  NAVIGATION  ANU  PUSIIiONInG  SYSTEMS* 

Emil  H.  .Tebe 
University  of  Michigan 

and 

Ralph  A.  King 
University  of  Wisconsin 


GLOSSARY  AND  DEFINITIONS 

1.  Analysis  of  Variance  ~  A  statistical  technique  based  on  a  linear 
model  and  the  application  of  least  squares  for  subdivision  of  the  total 
variability  in  a  sample  into  components  specified  by  the  model. 

2.  ACE(S)  -  Along  course  error  of  the  system  position. 

3.  bg0  -  Slope  of  the  orthogonal  regression  line  describing  the  system's 
path  based  upon  external  position  data. 

4.  CRD  -  Completely  randomized  design,  the  simplest  type  of  experimental 
design  or  pattern  of  experimentation.  The  treatment  combinations  are 
randomly  assigned  to  the  entire  set  of  experimental  units. 

5.  CCE(S)  -  Cross-course  error  of  the  system  position. 

6.  CCE(SP)  -  Cross-course  error  based'  upon  the  system’s  estimate  of  its 
own  position. 

7.  CCE(E)  -  Cross-course  error  (external)  »  length  of  a  perpendicular 
from  point  (X£,  Y£)  to  the  programmed  path  as  determined  by  external 
measurements. 

8.  Correlation  -  A  measure  of  linear  association  between  two  random 

variables:  o  »  °  /o  a  ,  i.e.,  a  ratio  of  the  covariance  to  the  product 
xy  x  y 

of  the  standard  deviations.  Sample  estimator,  r  ■  s  /s  s  . 

r  ’  xy  x  y 

9.  Chi  Square  Distribution  -  The  probability  distribution  of  the  square 
of  a  standard  normal  variable.  Let  be  N(0,i),  then  z  2  has  Chi  Square 
distribution  with  one  degree  of  freedom. 

*This  paper  also  was  presented  by  the  senior  author  at  the  "Technical 
Symposium  on  Navigation  and  Positioning,"  23-25  September  1969, 

USAECOM,  Fort  Monmouth,  New  Jersey. 


10.  Ax  =*  X_  -  X  -  deviation  of  the  system's  indicated  position  (X 

O  L, 

coordinate  from  an  external  measure  of  system's  location. 

11.  Ay  -  Y  -  Y_  *  deviation  of  the  system's  indicated  position  (Y 

b  L 

coordinate)  from  an  external  measure  of  system's  location. 

2  2  1/2 

12.  d  ■  radial  error  *  [(Ax)  +  (Ay)  ]  **  straight  line  distance  from 

system's  indicated  position  to  position  determined  by  external  measuring 
equipment.  Note  that  if  Ax  and  Ay  are  normal  random  variables,  then 

2  2 

do  is  distributed  as  Chi  Square  with  two  degrees  of  freedom. 

13.  Duplicate  *  A  subsample  of  an  experimental  unit;  one  of  two  measures 
of  system  performance  for  the  same  experimental  unit. 

14.  Degrees  of  freedom  «  Formally,  a  parameter  of  the  Chi  Square 
probability  distribution.  In  application,  the  number  of  independent 
deviations  available  for  estimating  a  variance  or  mean  square. 

15.  Experiment  -  Study  of  system  performance  over  a  set  of  experimental 
units . 

16.  Experimental  error  «  A  mean  square  or  quadratic  measure  of  system 
variability  about  its  average  performance  measured  over  a  set  of  homo¬ 
geneous  experimental  units. 

17.  Experimental  unit  ■  A  period  or  segment  of  system  operation  for 
which  an  independent  measure  of  system  performance  can  be  obtained. 

18.  F  ratio  -  A  ratio  of  two  independent  estimates  of  variance  for  which 
under  the  "null  hypothesis"  both  numerator  and  denominator  are  distributed 
as  (Chi  Square)  (t>2)  with  degrees  of  freedom,  say  f^  and  f ^ . 

19.  Interaction  -  A  situation  In  which  the  observed  results  for  the 
simultaneous  application  of  two  or  more  factors  cannot  be  explained  by 
addition  of  the  direct  effects  of  each  factor  separately  estimated. 

20.  Local  Control  -  (Blocking)  a  subdivision  of  the  total  set  of  available 
experimental  units  into  relatively  homogeneous  subsets.  Each  subset  is 
called  a  block.  A  complete  block  contains  one  experimental  unit  for  each 
treatment  combination.  Two  or  more  such  blocks  then  comprise  a  RCB. 

21.  Median  -  A  value  which  divide*  a  population  or  a  sample  into  two  equal 
parts. 

22.  Orthogonal  regression  line  -  A  least  squares  fitted  line  such  that 
the  sum  of  squares  of  normal  distances  from  points  to  the  line  is 
minimized. 
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23.  Position  error  (refer  text) 

24.  Quartile  -  The  quartiles  are  values  that  divide  a  population  or  a 
sample  into  four  equal  parts.  The  second  quartile  is  the  median. 

25.  Radial  error  •  d  (see  12  above). 

26.  RCB  -  Randomized  complete  block  design  (see  20  above). 

27.  Response  -  A  measure  of  system  performance.  May  be  univariate  but 
is  often  multivariate. 

28.  Replication  -  In  a  simple  measurement  situation  a  single  Independent 
observation  of  system  performance.  Otherwise,  one  replicate  comprises 
one  observation  of  system  performance  for  each  treatment  combination  of 
the  entire  Bet  of  treatment  combinations  being  investigated. 

29.  Regression  mean  square  -  The  mean  square  of  deviations  of  points  from 
a.  regression  line  (based  upon  division  by  the  degrees  of  freedom). 

30.  Sample  size  -  The  number  of  observations  on  each  treatment  combination 
or  the  number  of  complete  replicates.  Note:  this  is  not  the  total  number 
of  experimental  units. 

31.  Standard  deviation  -  Square  root  of  the  variance;  a  measure  of 
variability  about  the  average  of  observations  from  a  homogeneous  set  of 
experimental  units. 

32.  Structure  of  a  Test  Program  or  Experiment  -,The  overall  arrangement  of 
a  test ^program  which  includes  the  treatment  combinations  to  be  investigated, 
the  environments  an4  locations  in  which  the  system  is  to  be  operated  and 
the  experimental  design  imposed. 

33.  Treatments  (and  Treatment  Combination)  -  If  a  system  is  to  be  tested 
at  altitudes,  aay  Low  and  High,  we  sjay  that  altitude  la  a  factor  at  two 
levels.  We  also  refer  to  altitude  as  a  treatment  imposed  on  the  system. 
Suppose  we  also  wish  to  teat  the  system  over  land  and  over  water.  Then 
water  and  low  altitude  and  water  and  high  altitude  are  two  different 
treatment  combinations.  Two  factors  each  at  two  levels  provide  a  total 
of  four  treatment  combinations. 
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Field  Test  programs  are  fraught  with  many  difficulties.  Developmental 
equipment  just  never  seems  to  perform  as  well  as  desired  by  its  producers 
or  as  hoped  for  by  the  Army.  Characteristics  of  the  field  environment 
may  not  have  been  adequately  anticipated  by  the  development  engineers. 

Often  an  extensive  shakedown  period  is  required  before  a  system  is  really 
ready  to  be  entered  into  a  field  test  program. 

Even  before  the  shakedown  trials  are  started  a  complete  TEST  PLAN 
must  be  developed  for  the  field  test  program.  The  field  test  envisaged 
may  comprise  several  parts  with  each  part  designed  to  exercise  the  system 
in  a  different  way.  When  this  is  the  case,  a  specific  TEST  PLAN  should  be 
developed  for  each  part.  ^ 

It  has  been  our  experience  that  field  test  programs  are  often  \ 
Inadequate  or  incomplete  in  several  respects.  Therefore,  we  need  to 
consider  the  question,  "What  are  the  GENERAL  FEATURES  OF  A  TEST  PROGRAM?" 
These  features  are  set  out  as  a  list  of  ten  items  (prepared  by  the  senior 
author  at  a  time  when  he  first  came  in  contact  with  the  study  of  naviga¬ 
tion  and  positioning  systems)  [lj. 


GENERAL  FEATURES  OF  A  TEST  PROGRAM 

1.  Careful  delineation  of  the  problem  and  thorough  understanding  of  the 
system  or  systems  to  be  examined. 

2.  Definition  of  the  phenomena  to  be  studied.  (Including  "What  are  the 
requirements?") 

3.  Selection  of  the  response  (i.e.,  performance  characteristics)  and 
the  technique  of  measurement  for  each  response.  Know  the  standards 
that  should  be  applied. 

•i 

A.  Determination  of  a  suitable  experimental  unit. 

5.  Selection  of  treatments  to  be  studied  (i.e.,  equipment  parameters 
to  be  varied. 

6.  Selection  of  environmental  conditions  or  parameters  to  be  varied. 

7.  Choice  of  a  pattern  of  experimentation  (suitable  combinations  of 
experimental  units,  treatments  and  environment).  Result  is  an 
experimental  plan  or  design  that  Includes  the  randomization  pro¬ 
cedures,  adequate  local  controls  and  sufficient  replication. 

8.  Complete  layout  of  the  plan  for  analysis  of  the  responses  or 
measurements  to  be  obtained  (before  the  data  are  taken). 


9.  Interpretations  to  be  made  from  ail  possible  experimental  results. 

10.  What  is  the  next  experiment  that  may  be  relevant  after  the  currently 
proposed  one  is  completed? 


Let  us  consider  these  ten  items  In  turn.  Item  1,  we  leave  to  the 
engineers  although  much  questioning  is  often  required  to  obtain  a  clear 
statement  of  the  problem.  Item  2,  we  also  regard  largely,  as  an  engineering 
area.  Spelling  out  what  is  expected  of  the  system  in  realistic  and  use¬ 
ful  terms  is  a  major  step.  Later,  the  question  is  to  be  asked  and  answered, 
"Does  the  system  fulfill  the  requirements?"  To  a  considerable  extent  the 
answer  will  depend  on  the  data  acquired  and  our  analysis  of  these  data. 
Examples  of  requirements  might  be,  "Take  off  from  Dulles  International 
for  Paris;  make  landfall  in  France  with  cross-course  deviation  less  than 
5  miles  with  respect  to  a  designated  point."  Or,  "Take  off  from  Field  A; 
fly  over  point  X,  Y  with  an  average  radial  error  not  to  exceed  20  meters; 
land  at  Field  B." 

Succeeding  items  on  the  list  lead  us  more  into  the  statistical  and 
experimental  design  problems.  Determination  and  definition  of  the  relevant 
response  (Item  3)  for  judging  the  performance  of  the  system  is  basic  to  all 
that  follows.  Yet  many  "test  programs"  have  been  written  without  having 
the  performance'  measures  for  the  system  quantified  and  the  methods  of 
measurement  clearly  stated.  Related  to  the  performance  measure  is  the 
selection  of  the  standard  for  assessment  of  that  performance.  With  respect 
to  navigation  and  positioning  systems  we  may  ask, 

(1)  Do  we  need  photo-theodolite  data?,  or 

'4 

(2)  Is  a  radar  network  required?,  or 

(3)  Will  the  measurements  from  a  single  radar  such  as  the  FPS-16 
be  sufficient?,  or 

(4)  Can  we  rely  on  a  higher  resolution  non-radar  electronic 

network?,  or 

* 

(5)  Will  cruder  methods,  simple  photographs  or  visual  observation, 
be  sufficient? 

Depending  on  the  stated  requirements,  we  may  select  one  or  more  of 
these  alternatives. 

After  much  though  about  item  4,  we  have  reached  the  conclusion  that 
for  an  electronic  system  mounted  in  a  land  vehicle,  a  ship  or  an  A/C,  the 
‘entire  mission  on  a  given  day  must  be  regarded  as  the  experimental  unit. 

In  this  mission  we  include  -  " 
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starting  up  the  system 

warm-up 

check,  uul 

calibrations 

departure  from  base 

calibrations  enroute 

traversing  selected  courses 

return  to  base 

checking  calibrations 

shutting  down  the  system 

complete  return  to  ambient  conditions. 

This  view  of  the  experimental  unit  means  that  any  repetition  maneuvers 
performed  by  the  system  within  the  same  mission  must  be  regarded  as 
duplicates  and  not  as  replicates.  Of  course,  we  are  interested  in  the 
variation  among  duplicates  but  major  Interest  centers  on  the  replicates, 
that  is,  the  repeated  performance  of  the  system  over  a  set  of  experi¬ 
mental  units  that  we  regard  as  similar  or  sufficiently  homogeneous  for 
the  problem  under  study.  By  definition,  experimental  error  is  the 
failure  of  a  system  to  produce  identical  responses  over  a  set  of  in¬ 
dependent  trials  (or  experimental  units) .  The  key  word  here  is 
Independent ;  we  believe  that  repeated  maneuvers  in  any  one  mission  are 
likely  to  be  highly  correlated.  Therefore,  we  insist  that  an  independent 
trial  for  an. electronic  system  include  the  complete  sequence  given  above 
from  "starting  up  the  system"  through  "return  to  ambient  conditions." 

A  system  may  have  several  "modes"  of  operation,  threshold  settings 
may  be  required  and  variation  of  dial  settings  may  affect  the  performance 
of  the  system.  All  these  equipment  parameter  variations  we  include  under 
the  set  of  treatments  that  may  be  investigated  (item  5).  Further,  we 
usually  extend  oUr  concept  of  the  treatments  of  Interest  to  include  the 
variations  external  to  the  system  which  may  or  may  not  affect  (hopefully 
not)  the  performance  of  the  system.  Under  Item  6,  we  Include  weather, 
altitude,  day  or  night  operation,  electromagnetic  disturbances  (natural 
or  man-made),  terrain,  direction  qver  a  course,  etc. 

The  result  of  considering  Items  4,  5,  and  6  leads  us  to  selection  of 
a  pattern  or  program  for  the  system  test.  The  structure  of  the  test  pro¬ 
gram  Is  determined  by  the  factors  (conditions  and  parameter  settings)  which 
we  wish  to  investigate.  The  simpler  this  structure  can  be  made,  the 
easier  it  will  be  to: 

• 

(1)  Cope  with  the  inevitable  Modifications  of  the  test  program 
that  arise  due  to  revision  of  test  objectives,  unexpected 
equipment  limitations,  or  failure  to  obtain  adequate  data 
for  some  courses;  and, 

(2)  Analyze  the  data. 
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Randomized  Design  (CRD).  This  design  is  preferred  when  it  is  feasible. 

A  simple  description  is  that  we  write  down  on  slips  of  paper  each  com¬ 
bination  of  conditions  and  parameter  settings  that  is  to  be  included 

in  the  test  program.  Then  we  put  the  slips  into  a  hat,  mix  thoroughly, 

draw  them  out  one  at  a  time  and  write  out  a  complete  list  of  the  con¬ 
secutive  drawings.  Suppose  altitudes  of  one  thousand  and  12,000  feet 
were  Included  in  the  test  program  for  an  airborne  system.  If  any  part 
of  the  consecutive  sequence  of  drawings  came  out  with  altitudes  (in 
thousands  of  feet)  12,  1,  12,  1,  12  for  the  sequence  of  courses  to  be  ' 

flown  the  pilots  would  object;  hence,  we  regard  a  CRD  as  not  feasible 
for  such  a  situation.  Therefore,  split-plot  structures  or  nested  designs 
must  be  worked  out  when  some  of  the  treatments  cannot  be  submitted  to 
complete  randomizations. 

Performance  of  systems  tends  to  vary  with  time,  or  for  a  development 
item  prototype  the  performance  is  even  likely  to  deteriorate  with  time. 

Such  results  are  to  be  expected  when  the  "bugs"  are  not  all  ironed  out, 
and  the  test  program  covers  a  3  to  6  month  period.  Because  of  this  time  / 

variability  in  performance,  it  is  highly  desirable  to  introduce  a  "blocking’ 
with  respect  to  time.  Such  blocking  la  a  form  of  what  is  generally  known/7 
in  experimental,  design  as  "local  control*"  This  local  control  permits  / 
the  removal  of  (or  elimination)  of  time  variation  so  that  any  two  treat/- 
ment  combinations  (choice  of  parameter  settings)  can  be  compared  without 
time  bias.  What  this  means  in  practice  is  that  if  two  particular  combina¬ 
tions  are  run  in,  say,  the  second  week  of,  the  test  program,  and  if  one  or 
the  other  is  scheduled  again  for  the  7th  and  13th  weeks  of  the  test  program, 
then  if  both  are  run  in  the  7th  and  13th  week,  then  the  time  differences 
(if  any)  aipon^  the  2nd,  7th,  and  13th  weeks  can  be  removed  in  making  the 
desired  comparison.  The  balancing  of  the  experimental  program  against 
time  or  some  other  possible  source  of  undesirable  variability  is  accomp¬ 
lished  by  setting  up  a  Randomized  Complete  Block  design.  We  regard  the 
use  of  local  control  by  blocking  as  a  necessary  requirement  in  the  study 
of  complex  systems  used  for  navigation  and  position  determination.  Here, 
we  have  assumed  one  week  as  comprising  a  block. 

it  is  to  be  noted  that  each  block  as  just  ^escribed  forms  one  complete 
replicate  for  a  set  of  treatment  combinations.  The  time  period  included 
in  the  block  can  be  any  reasonably  short  period  of  homogeneous  test  con¬ 
ditions,  say,  one  day,  three  days,  or  one  week.  Thus,  the  number  of  blocks 
completed  determines  the  total  number  of  replicates  for  this  set  of  treat¬ 
ment  combinations.  The  number  of  blocks  completed  then  determines  the 
sample  size  so  the  natural  question  is,  "How  many  blocks  do  we  need?" 

Two  considerations  enter  into  the  determination  of  the  desired  sample 
size.  First  is  the  requirement  of  obtaining  a  stable  estimate  of  the 
experimental  error.  It  ia  our  experience  that  an  estimate  with  10  to  20 
degrees  of  freedom  may  of tan  be  adequate  for  development  test  programs. 

Such  an  estimate  can  be  obtained  with  as  few  as  three  blocks  when  eight 
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or  more  treatment  combinations  are  to  be  investigated  in  each  block. 
Larger  blocks,  however,  may  introduce  other  problems;  e.g.,  lack  of 
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magnitude  of  real  differences  in  system  performance  that  may  be 
associated  with  environmental  and/or  parametric  changes  for  the 
system.  Again  from  experience  we  have  found  that  system  developers 
and  system  users  have  limited  information  on  the  magnitudes  of  these 
differences.  It  can  be  shown  from  theory  that  for  a  specified  prob¬ 
ability  a  "large"  sample  is  required  to  detect  "small"  differences, 
but  that  a  "modest"  sample  may  detect  easily  "large"  differences. 

These  vague  words  (large,  small,  modest)  can  be  given  numerical  values 
only  when  we  are  able  to  insert  in  the  available  formulae  actual  values  ^ 
for  (1)  the  standard  deviation  of  our  experiment  the  (experimental  error)1 
previously  described);  and  (2)  the  magnitude  of  the  difference  to  be 
detected. 


The  discussion  of  Item  7  of  the  "General  Features"  has  been  rather 
Lengthy,  but  we  have  tied  together  in  this  discussion  the  preceding 
Items  4,  5,  and  6  with  Item  7.  In  this  discussion  we  have  coyered  some 
aspects  of  the  choice  of  experimental  pattern  and  its  associated  randomiza¬ 
tion,  local  control  by  blocking  on  time,  and  the  choice  of  sample  size. 

Item  8  follows  quite  easily  If  we  have  done  our  homework  well  in 
covering  Items  3  through  7.  Perhaps,  we  should  note  that  it.  is  easy 
only  in  principle.  We  recall  a  paragraph  from  our  abstract  as  follows: 

When  these  'GENERAL  FEATURES'  have  been  closely  adhered 
to,  then  the  work  of  summarization  and  analysis  of  data  and  the 
final  Interpretation  of  results  becomes  much  simpler.  An  ex¬ 
perimental  design  for  the  field  test  program  haa  associated  with 
it  a  mathematical  model;  the  two  together  determine  the  analytical 
procedures.  One  of  the  most  useful  and  severe  disciplines  to 
impose,  on  the  military  personnel  and  the  development  contractor 
is  to  require  that  a  set  of  tables  be  prepared  before  the  field 
test  is  started.  This  set  of  tables  s7iould  include  the  detailed 
format  of  the  summary  data  on  which  the  performance  of  the  system 
is  to  be  judged.  Further,  the  parties  should  agree  that  the 
performance  is  to  be  judged  on  these  criteria. 

The  last  two  Items,  9  and  10,  are  essentially  self-explanatory.  It 
is  usually  salutary  to  give  them  some  consideration,  however,  before  the 
first  experiment  is  begun.  As  the  test  program  proceeds,  other  considera¬ 
tions  will  appear  or  come  to  bear  on  the  problem.  Thoughts  about  9  and  10 
will  then  take  new  directions.  Without  the  pre-first-experiment  considera¬ 
tions  well  though  out  and  written  down,  the  new  directions  may  turn  out  to 
be  undesirable  tangents.  The  "whole  forest  needs  to  be  kept  in  view  rather 
than  the  Interesting  trees  that  appear  as  we  walk  in  the  woods."  A  remark 
on  the  use  of  the  term  experiment  may  be  added  here.  Physical  scientists 
often  think  of  an  experiment  as  a  single  trial  under  carefully  specified 
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conditions.  From  the  analysis  point  of  view,  which  must  be  taken  bv 
cne  statistician,  an  experiment  comprises  all  replicates  of  a  set  of 
treatment  combinations  among  which  comparisons  are  to  be  made.  The 
TEST  PLAN  (or  test  program)  for  a  given  system  may  consist  of  only 
one,  or  two  or  more  experiments.* 

We  now  turn  to  the  consideration  of  the  second  area  of  our  paper 
as  indicated  by  our  title.  Analysis  of  the  performance  of  a  navigation 
and  positioning  system  must  describe  this  performance  quantitatively  in 
terms  of  precision  and  accuracy  [5].  Various  statistical  techniques  may 
be  required  to  describe  this  performance.  In  order  to  give  concreteness 
to  this  section  of  the  paper  wa  shall  base  our  discussion  upon  the 
analysis  of  the  performance  of  an  airborne  navigation  system  in  which 
we  were  engaged  several  years  ago  [6]. 

The  field  test  program  for  this  system  included  a  requirement  that 
the  system  depart  from  a  base,  fly  over  a  calibration  check  point,  and 
then  proceed  to  maneuver  the  A/C  over  a  series  of  six  parallel  flight 
paths  whose  end  points  were  defined  by  specified  longitude  and  latitude 
coordinates  (see 'Figure  1).  In  Figure  1,  we  show  two  series  of  six 
parallel  lines,  sets  LA  and  IB.  The  set  IB  was  actually  laid  over  the 
same  ground  area  as  set  1A.  Each  line  of  a  set  of  six  we  refer  to  as  a 
LEG,  so  that  the  total  flight  course  comprised  six  LEG'S.  Starting  with 
LEG  1  in  series  1A  as  shown  we  refer  to  this  pattern  as  a  Zero  Degree 
flight  forward  over  the  course  (AO,F).  Beginning  with  LEG  6  and  reversing 
direction  over  each  LEG  Is  called  (AO,  R).  Using  series  IB  in  the  direc¬ 
tion  shown  starting  with  LEG  1  is  designated  as  90  degrees  forward  (B90, 
F).  Similarly,  reversing  course  beginning  with  LEG  6  Is  designated  as 
(B90,  R).  Other  designations  are  possible  such  us  starting  at  other  end 
of  LEG  1  in  each  aeries,  which  gives  (A180,  F)  and  (B270,  F) . 

With  this  view  of  the  flight  area  pattern  we  may  approach  the  details 
of  describing  the  system  performance.  Assessment  of  the  system  perfor¬ 
mance  will  be  based  largely  on  a  position  error;  i.e.,  the  difference 
in  location  of  the  system  as  determined  by  an  external  measuring  system 
and  the  system's  own  indication  of  Its  location  (at  a  given  time).  This 
position  error  information  Is  to  be  analyzed  by  averaging  and/or  decom¬ 
position  to  provide  descriptions  of  system  performance.  Among  these 
descriptions  are: 

(1)  The  difference  between  the  average  location  of  the  system 
over  a  number  of  repetitions  under  essentially  similar 
conditions  for  a  programmed  flight  over  a  point  or  a  course 
and  the  desired  point  or  course  is  a  measure  of  system 
accuracy  [7].  This  accuracy,  however,  may  vary  over  the 
flight  area  (1A  &  IB)  for  a  variety  of  reasons.  Thus,  it 
may  be  useful  to  speak  of  the  system's  predictability  or 
reproducibility  for  a  group  of  points  or  LEG's  in  the 
assessment  of  accuracy, 


^Appropriate  references  for  this  first  section  of  the  paper  are  [2],  [3], 
and  [ 4 ] . 
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(2)  For  precision  or  repeatability  assessment  we  may  describe 
the  performance  along  a  specific  LEG  or  segment  of  a  course 
which  was  programmed  for  the  system,  or 

(3)  We  may  describe  the  repeatability  of  the  system  in  flying 
over  the  same  programmed  course  a  number  of  times,  each 
time  appearing  on  a  different  day. 

Thus,  it  is  seen  from  (2)  and  (3)  that  we  can  describe  precision 
over  experimental  units  (replicates),  which  is  of  greatest  interest,  and 
also  in  terms  of  within  replicates  over  a  segment  of  a  LEG,  a  whole  LEG 
or  the  set  of  LEG'S.  Replications  of  any  LEG  or  part  of  a  LEG  on  the 
same  day  may  be  regarded  as  duplicates  from  the  viewpoint  of  sampling 
the  system  performance.  We  note  that  within  a  single  programmed  flight 
on  the  same  day  all  individual  position  determinations  made  by  the 
system  must  be  regarded  as  inherently  correlated  to  some  greater  or 
lesser  but  unknown  degree.  This  point  of  view  is  conceptually  correct 
in  regarding  the  output  of  a  single  programmed  flight  as  one  "realiza¬ 
tion"  in  the  sense  of  the  theory  of  stochastic  processes.  '  The  degree 
of  correlation,  of  course,  depends  on  the  time  and/or  distance  separa¬ 
tion  between  any  two  position  determinations.  The  actual  magnitude  or 
form  and  shape  of  this  correlation  function  may  be  quite  relevant  for 
system  design  but  need  not  be  of  major  concern  for  evaluation  of  system 
performance.  The  realization  of  its  presence,  however,  requires  the 
definition  of  a  single  trial  or  experimental  unit  in  the  way  already 
described  and  then  it  guides  our  analysis. 

The  discussion  thus  far  has  been  general  in  the  evaluation  of 
system  performance.  It  will  be  helpful  to  list  some  of  the  actual 
variables  measured  in  relation  to  the  determination  of  position  error. 
These  random  variables  were: 

(1)  Ax  -  Xs  -  XE 

<2>  Ay  ■  Ys  -  Ye 

(3)  CCE(S)  *  Cross-course  error  for  the  system 

(4)  ACE(S)  -  Along  course  error  for  the  system 

2  2  1/2 

(5)  d  -  [(Ax)  +  (Ay)  ]  '  ■  Radial  error  for  the  system. 

A  rectangular  grid  system  was  laid  out  over  the  area  indicated  in  Figure  1 
with  the  point  (0,0)  arbitrarily  selected.  At  time  t . ,  (S  ,  Y  )  was 

X  81  wl 

the  system's  indicated  position  while  (Xp, ,  Y  )  was  the  actual  position 
of  the  system  as  determined  by  an  external  means.  Thus,  was  the  radial 
error  at  time  t^.  The  time  interval  from  t^  to  t^  +  1  was  five  seconds. 


359 


The  assessment  of  repeatability  is  most  easily  begun  by  examining 
the  performance  within  a  single  LEG.  For  each  LEG  a  number  of  summary 
statistics  were  computed  for  the  variables  just  listed.  These  statistics 
included: 


(1)  Average  value  for  the  variable. 

(2)  Mean  square  deviation  of  the  individual  values  from  the 

average.  Note  that  this'  quantity  although  calculated  like 
a  sample  variance  does  not  have  the  usual  Chi  Square  dis¬ 
tribution  with  n  -  1  degrees  of  freedom  because  of  the 
correlation  of  data  points  within  a  given  LEG  (as  already 
discussed) . 

(3)  Minimum  value. 

.  i 

(A)  Maximum  value. 

| 

(3)  Median  value.  | 

\  i 

(6)  First  and  third  quartiles. 

In  this  paper  we  can  illustrate  only  a  few  analyses  of  these  many 
statistics.  A  mere  tabular  summary,  of  course,  gives  some  description 
of  repeatability.  A  further  analysis  considers  the  behavior  of  these 
LEG  statistics  from  LEG  to  LEG,  from  (programmed)  flight  to  flight,  at 
different  altitudes,  orientations  (or  direction  of  flight),  and  even 
over  different  areas.  The  statistical  technique  used  for  this  further 
analysis  is  known  as  the  analysis  of  variance.  This  technique  has  been 
well  described  by  Kempthorne  and  Scheffe*  in  its  application  to  the 
analysis  of  experimental  data  [8,  9].  Briefly,  the  technique  may  be 
described  as  a  procedure  for  evaluating  the  variation  of  averages  and 
the  variation  of  individual  observations.  These  evaluations,  called 
mean  squares,  may  be  compared  by  forming  Snedecor's  F  ratio  in  order 
to  make  inferences  about  the  magnitude  of  the  variations  of  the  averages. 
Specific  assumptions,  of  course,  are  made  in  the  application  of  the 
technique.  Currently,  most  attention  is  given  to  these  assumptions: 

(1)  the  specified  linear  model  adequately  represents  the  experimental 
structure;  and,  (2)  independence,  i.e.,  the  data  comprise  a  random 
sample  from  the  universe  of  interest. 

A  simplified  example  will  illustrate  the  application  of  the  analysis 
of  variance  to  a  possible  set  of  data  from  the  flight  program  described 
above.  Let  us  suppose  a  series  of  flights  made  over  the  area  of  Figure  1 
with  variations  in  altitude  and  heading.  The  series  of  flights  is  carried 
6ut  in  a  completely  randomized  design  with  the  results  obtained  as  in 
Table  1.  There  are  two  replicates  of  each  combination  of  conditions. 

Note  that  only  average  results  for  each  entire  flight  are  presented. 


The  analysis  of  variance  appears  In  Table  2. 
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TAHLE  1 


T  LL *  ’ T iwVrT  V  Y  XAJ e  lT  L 
FOR  ririHT  FLIOiiTK 
two  alt  it  or0 

*•  nn  nrifi  p  t  r  r-  r»nn»»  a»r 
,  »  '  1  1  » y  1 .  ’  1  •  >  » .  i\l .  l  t  <  I  •  * 

FOR  COM” I NATIONS  OF 

A':n  TWO  "KADIWC^ 

Flight  ''a. 

A1 titudc 

Heading 

(Degrees) 

Average  Radial 
Error  (Meters) 

1 

7,000 

n 

SO 

2 

7,000 

90 

90 

3 

Ir,000 

0 

50 

4 

7,000 

0 

90 

5 

15,000 

90 

40 

6 

7,000 

90 

100 

7 

15,000 

0 

50 

R 

15,000 

90 

60 

560 


I' A  I.  !* 


Analysis  of  Variance  of  Average  Radial  Error 


Source  of 

Variation 

Degrees  <0 

Freedom 

“uim  of 

Squares 

Mcnn 

Snuare 

Total 

/ 

O 

/,  2gnn 

---- 

Average 

l 

39300 

/ 

Altitude 

l 

3300 

3300 

Heading 

l 

50 

50 

Altitude  by 

Heading  Interaction 

1 

50 

50 

Remainder 

t. 

300 

75 

The  model  upon  which 

this  analysis  I s  based 

is  written  as  - 

Yi1k  -  b  +  At  + 

", +  <A">1(  *  hik 

* 

« 
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where  'ii  lL-  is  an  average  radial  error  as  shown  in  Table  1  and  the 
terms  on  the  right  in  order  are  -  . 

a  general  mean, 
an  altitude  effect, 
a  heading  effect, 

an  altitude-heading  Interaction,  and 

a  random  component  associated  with  the  ijk  th  experimental 
unit  (flight). 

It  is  the  variation  among  these  eight  averages  given  in  Table  1 
which  is  to  be  subdivided  into  parts  associated  with  tho  sources  of 
variation  present.  We  note  that  the  altitude  means  are:  1/4  (80490+ 
904100)  -  90  at  7,000',  and  1/4  (504404504*0)  -  50  at  15,000'. 
Similarly,  the  Heading  means  are:  67.5  at  0  degrees  and  72.5  at  90 
degrees . 

Thus,  the  2x2  table  of  means  for  average  radial  error  is 


Altitudes 


Headings 

7 

15 

Averages 

0 

85  ■' 

50 

67.5 

90 

95 

50 

72.5 

. 

Averages 

90 

50 

70 

Details  of  the  calculations,  the  assumptions  underlying  the  analysis  and 
interpretation  of  the  results  are  given  in  most  modern  texts  on  statis¬ 
tical  theory  or  techniques  [10,  11] .  We  cannot  consider  these  matters 
further  here ,  but  we  point  out  two  aspects  of  this  hypothetical  example: 
(1)  The  "Remainder"  with  4  degrees  of  freedom  is  an  appropriate  estimate 
of  experimental  error,  so  that  (75) 1/2  »  8.66,  is  a  standard  deviation 
that  estimates  the  repeatability  of  the  System  over  repeated  flights; 
and,  (2)  that  the  Mean  Square  for  Altitude,  3200,  when  compared  with  the 
Remainder  Mean  Square  provides  a  basis  for  assessing  the  effect  of 
Altitude.  If  Altitude  variation  did  not  effect  the  System,  we  would 
expect  these  Mean  Squares  to  be  about  equal.  From  Table  2  we  would 
conclude  by  looking  at  the  interaction  component  (F  ratio  ■  50/75(1) 
that  the  Altitude  effect  does  not  vary  with  Heading.  Tho  Heading  effect 


t 
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appears  negligible  (F  ratio  ■*  50/75  1).  Finally,  we  would  conclude 
that  performance  differs  with  altitude  (F  ratio  *  3200/75  -  43) 

(P  «■  <0.01).  From  the  averages,  we  see  that  the  radial  error  is 
much  smaller  at  the  higher  altitude. 

In  reference  to  the  description  given  above  for  the  analysis 
of  variance  as  a  technique  for  studying  the  variation  of  averages 
in  contrast  to  the  variation  of  individual  observations,  there  is 
a  point  to  be  noted  in  relation  to  the  hypothetical  example  just 
given.  In  the  example,  the  individual  values  analyzed  are  them¬ 
selves  averages.  Thus,  there  is  a  further  component  of  variation 
associated  with  individual  observations  or  points  within  LEGS, 
that  has  been  suppressed  in  the  example.  Generally,  in  analyzing 
data  for  studying  the  system  we  followed  this  same  procedure  of 
studying  averages.  Thus,  a  simple  LEG  average  provided  a  single 
datum  and  we  analyzed  the  variation  of  these  averages  in  relation 
to  other  factors. 

There  are  several  reasons  for  fallowing  this  procedure.  First, 
this  approach,  of  course,  has  simplified  some  problems  in  analysis  due 
to  unequal  numbers  of  observations  within  LEGS.  Second,  even  though 
numbers  of  observations  on  a  given  LEG  varied  from  as  low  as  80  to 
around  200,  there  was  no  reason  for  giving  more  weight  to  one  flight; 
over  a  given  LEG  than  another  if  a  reasonable  set  of  data  were  obtained 
to  represent  that  flight  over  that  LEG.  Thus,  using  averages  and  giVlng 
each  average  equal  weight  seemed  a  proper  procedure  for  assessing  the 
overall  performance.  Third,  the  use  of  averages,  even  though  each 
average  is  computed  from  data  with  considerable  correlation,  will  pro^- 
vide  values  of  a  random  variable  which  more  closely  approach  the  assump¬ 
tions  of  the  analysis  of  variance  technique.  In  analyzing  the  repeat¬ 
ability  within  LEGS  aa  measured  by  the  variances  of  designated  random 
variables  (Mean  Square  Deviations  from  Average  or  from  an  Orthogonal 
Regression  Line),  these  variances  may  also  be  considered  as  "averages." 
Because  of  the  greater  apparent  dispersion  of  these  variances,  it 
seemed  desirable  to  analyze  the  natural  logarithms  of  these  quantities 
to  obtain  a  transformed  variable  more  suitable  for  the  analysis  of 
variance  technique.  Fourth,  and  last,  this  approach  in  terms  of  further 
analysis  of  original  statistics  (averages,  variances,  slopes  of  regression 
lines  and  deviations  from  such  lines)  is  in  keeping  with  the  spirit  of 
Professor  John  W.  Tukey's  suggestions  (12]. 

The  preceding  example  was  made  small  in  order  to  be  easy  to  follow. 
The  conclusions  stated  relate  only  to  the  hypothetical  data  of  Table  1 
as  if  they  were  real  data.  We  now  present  in  Table  3  some  real  data  for 
six  flights  over  the  area  represented  by  Figure  1.  These  flights  were 
flown  at  three  altitudes  with  zero  degree  heading  (i.e.  ,  AO,  F  as  noted 
above).  Table  3  gives  averages  of  Ax,  Ay  and  d  -  radial  error  for  each 
LEG  of  each  flight.  Hence,  36  averages  are  shown  for  each  variable.  The 
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Table  1 


Tabulation  of  Leg  Means  for  Selected  Variable* 
f‘r,vi  Six  Flights  ftvor  the  ri.g-ir*  1  \rea 
(Unit*  are  Meters) 


Flight 

'lumber 


5 

6 
1 
7 

3 

4 


1 

f) 

1 

7 

3 

4 


(Altitude/ 

Lag  Number 

MHO) 

l 

2 

1 

* 

5 

6 

Variable: 

A<  -  %  - 

7.5 

14) 

12 

54 

too 

120 

116 

7.5 

10  1 

32 

51 

91 

li'8 

115 

11 

3:1 

-10.5 

-9 

51 

71 

9f, 

1  1 

05 

70 

74 

101 

124 

;  io 

13 

121 

81 

107 

Rfl 

125 

116 

15 

01 

.90 

50 

119 

07 

140 

Variable: 

Av  *  Vs  - 

V 

11 

7,  ; 

-70 

-15)5 

-333 

-341 

-144 

+11 

7.5 

-75 

-101 

-233 

-257 

-80 

+45 

1.1 

+  35 

-330 

-7" 

-J.V3 

+19 

+185 

'1 

-02 

-230 

-341 

-371 

-172 

+3 

1  5 

-100 

-1)9 

-V.0 

-124 

-05 

+3 

r> 

-103 

-275 

-131 

-391 

-122 

-3 

Varinbla: 

Radial  Krror  “  {(Ax)11 

+  Uy\ 

>V'2 

7.5 

319 

187 

349 

370 

219 

159 

267* 

7.  5 

27’ 

139 

2.11 

788 

19'. 

142 

216  . 

11 

278 

378 

176 

169 

168  0 

212 

230 

11 

765 

253 

136 

402, 

231 

175 

280 

15 

264 

186 

370 

34  4 

208 

138 

252 

15 

260 

508 

157 

412 

200 

196 

287 

277+*  240 

310 

331 

204 

170 

2-55*** 


*¥1  light  Averages; 

**Lag  Averagos:  t 

***Ovarall  Average. 
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.  tiMo  /, 

Analysis  Of  Variance  of  the  Leg  Averages 
for  the  Variable:  Radial  Frror 


Sotircp  of 
Varation 

Total 
Average 
Alt! tudes 

Flights  at  Same  Altitude 
(Pooled  Variation  for  Flights) 
Lega  Over  All  Flights 
Le®«  x  Altutudeg  Tntaractlons 
Lega  x  Flights  at  Same  Altitudes 
(Grouped  Legs  x  Flights) 


Hegrpes  of 

Freedom 

Sum  of 

Squares 

Mean 

Square 

1iS 

2582140 

f 

23461S6 

? 

A  77« 

238° 

■> 

6374 

(5)  . 

(250701 

(4774) 

,  S 

11*!  0.5.8 

23194 

10 

4oqno 

4030 

1? 

55847 

3723 

(25) 

(16147) 

0846) 

% 
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analysis  of  variance  for  one  of  the  variables,  radial  error,  is  given 
in  Table  4.  Therefore,  Table  3  also  shoes  the  marginal  averages  for 
this  variable,  that  is,  over  all  LEG's  of  the  same  flight,  and  over 
all  flights  for  the  same  LEG. 

The  model  for  the  analysis  of  Table  4  is  vritteu  as  - 


‘ijk 


+  + 


£ij  +  Lk  +  <AL> 


ik 


ijk 


where  Y  ^  1®  average  radial  error  as  given  in  Table  3  and  the  terms 
on  the  right  are  - 


a  general  mean, 
an  altitude  effect,  . 

an  error  component  for  flights  at  same  altitude, 
a  LEG  effect, 

an  ALTITUDE  x  LEG  interaction,  arid  ' 

a  residual  which  measures  failure  to  obtain  same  results  for 
a  LEG  when  a  repeated  flight  is  made  at  the  3ame  Altutude. 

Major  interest  in  Table  4  firsi:  centers  on  the  Altitude  Comparison. 
The  mean  square  for  Altitudes  is  23(89  while  the  mean  square  for  repeated 
flights  made  at  the  same  altitude  is  5364.  The  latter  is  our  measure 
of  experimental  error  for  Altitudes;  hence,  the  F  ratio  is  2389/6364<  1. 
We  conclude  that  altitude  variation  did  not  affect  the  performance  of 
the  system  ever  the  rahge  of  altitudes  selected  (our  choice  of  altitudes 
was  limited  by  the  performance  capability  of  the  A/C  carrying  the  naviga¬ 
tion  system) . 

Next ,“  we  examine  the  variation  within  flights  or  between  LEGs.  The 
"LEGb  over  all  flights"  mean  square  is  23194,  a  large  value  relative  to 
all  other  mean  squares  in  Table  4.  Thus,  we  are  inclined  to  conclude 
that  there  ere  large  differences  among  the  six  LEGS  of  the  programmed 
flight  pattern.  The  remaining  two  mean  squares,  LEGs  x  Altitudes  »  4030 
and  LEGs  x  Flights  at  same  Altitude  ■  3723,  indicate  the  consistency  of 
these  large  LEG  differences.  Pooling  of  the  last  two  sources  of  varia¬ 
tion  yields  a  mean  square  of  3846  with  25  degrees  of  freedom.  An  approxi 
mate  F  ratio  for  comparing  LEGs  could  be  formed  by  F  »  23194/3846  *  6. 

We  regard  thlB  ratio  as  an  approximate  F  in  distribution  because  of  the 
correlation  of  LEGs  within  the  same  flight  although  this  may  be  small 
because  of  the  apparently  large  LEG  differences.  Perhaps,  a  multivariate 
test  could  be  devised  for  comparing  LEGs;  we  have  not  considered  this 
approach.  In  view  of  the  consistency  of  these  LEG  differences  over  dif¬ 
ferent  days  throughout  the  test  program  it  seemed  reasonable  to  us  to 
conclude  that  natural  electromagnetic  field  variations  over  the  six  LEGs 
affected  the  system  performance. 
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Uo  «Ha1 1  not  An/iaau/ir  f*fff  a  nanor  hn  aiitnmarl  7a  mtr  rnnnliia^  Ana 

about  the  repeatability  or  precision  of  the  system  which  produced  the 
data  given  in  Table  3.  It  is  our  purpose  here  merely  to  present 
statistical  methods  and  techniques  for  securing  such  Information  about 
any  navigation  system.  It  will  be  useful  to  give  one  more  table,  how¬ 
ever,  to  show  another  aspect  of  the  repeatability.  In  Table  5,  we  give 
the  mean  square  deviation  of  the  radial  errors  from  the  average  radial 
error  (given  in  Table  3)  arranged  by  Flights  and  LEGs  as  in  Table  3. 

We  shall  not  give  the  analysis  of  variance  for  the  data  in  Table  5  but 
we  note  that  natural  logarithms  of  these  mean  square  deviations  were 
taken  before  computing  the  analysis  of  variance.  This  log  transforma¬ 
tion  is  usually  applied  before  analyzing  variances  of  observations. 

Although  we  have  given  only  a  small  sample  of  the  large  amount  of 
repeatability  information  obtained  for  the  system  we  have  been  using 
Hfor  qur  discussion,  we  turn  now  to  the  system  accuracy.  If  the  system 
exhibits  accurate  performance  we  may  say  that  it  has  predictability  or 
reproducibility.  In  addition  to  the  variables  listed  above,  which, 
were  used  for  examining  the  system  repeatability,  we  also  obtained  the 
cross  course  error  of  the  system  location  from  the  external  measure¬ 
ments,  CCE(E) ,  which  was  ttib  distance  of  the  point  (Xg,  Yg)  from  the 

programmed  path  (lines  shown  in'  Figure  1) .  :  Art  average  of  these  values 
would  show  the  bias  or  systematic  error  of  the  system  in  flying  the 
programmed  course.  If  this  bias  were  negligible  over  all  flights  we 
would  regard  the  system  as  accurate  or  that  its  performance  is 
reproducible. 

Further,  from  the  (Xg^ ,  Yg^)  data  we  obtained  a  derived  quantity, 

the  slope  of  the  orthogonal  regression  line,  bgg,  through  the  points 

traversed  by  the  system.  The  slope  of  this  line  for  each  LEG  of  the 
programmed  path  then  could  be  compared  with  the  actual  slope,  8,  of 
the  programmed  path  in  terms  of  the  arbitrary  X,  Y  coordinate  system 
imposed  on  the  area  of  Figure  1.  Departures  of  the  observed  slopes, 
bgQ  from  the  desired  slope,  8,  then  give  further  Information  on  the 

system  predictability. 

/ 

Before  presenting  actual  results  it  will  be  helpful  to  discuss 
briefly  the  use  of  the  orthogonal  regression  line.  From  the  above, 
it  is  clear  that  both  Xg  and  Yg  are  random  variables.  The  usual 

regression  models  consider  Xg  to  be  au  Independent  variable  observed 

or  measured  with  no  error  or  negligible  error.  Natrella  in  "Experimental 
Statistics"  gives  a  good  discussion  of  regression  analysis  for  functional 
and  statistical  relations  [13],  It  is  clear  that  none  of  the  standard 
models  apply  to  thio  navigation  system  analysis.  After  considerable 
study,  we  concluded  that  fitting  the  orthogonal  regression  line  would 
beat  describe  the  system  performance.  Derivation  of  the  normal  equa~ 
tions  for  fitting  this  line  was  given  by  Coleman  iu  1932  [14].  To  our 
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TAJiLF.  5 


Mean  Square  Devlatione  by  LF.Og  for  Six  Flights 
Variable:  Radial  I'rror 

LEG 

Flight 


Ntjn.be  r 

Altitude 

1 

v  i 

3 

4 

5 

6 

5 

7.5 

51240 

15630 

3  2740 

5215 

12720 

6883 

6 

7.5 

41050 

.  7*54 

26730 

5716 

7335 

1017 

1 

11 

22770 

155100 

16Vf«i 

4622 

3332 

6394 

7 

11 

19400 

27780 

13510 

6901 

12R90 

4270 

3 

IV, 

36400 

21097 

?6?on 

0261. 

5177 

1227 

4 

15 

14170 

28830 

!  25470 

1196 

7304 

8589 

V 
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TA’  I,''  6 


Tabulation  of  Altitude  Averages  and  General 
Average  for  Variables  ''hich  describe  System  Predlctahl |i tv 
(based  on  Li-'G  AV;  Pw»\<-!‘S  weighted  eouallv) 


Variable 

Genernl 

Average* 

(units  are  meters)  ^ 

Altitudea/fioori)  (feet) 

7.5  1) 

AV'HAGiS 

13 

RCT(K) 

-20,-.! 

-7.4 

-?2. 5 

••  30.  R 

CCF(S») 

r.i 

10.0 

7.0 

2.2 

u  it  irk 

slope,  b,,0 

-1.3001 

-i .  JO®.;; 

-1.30  30 

-1 . 2900 

Rag.  '<ean 

Square 

5«on 

3300 

7600 

A200 

Maaed  on  average  of  36  values;  six  flights  of  six  LFT.x. 
Mased  on  average  of  12  values;  two  flights  of  nix  t.FGn. 
***  Note:  «  -l.Tvr 
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Tabulation  of  Flight  Averages  for  Variables 
’dutch  hescrihe  Svateri  vredlrtnMI  I  tv 

i  • 

Chased  on  Lug  Averages  weighted  eoimUO 
fimit*  ire  notersi 
Altitudes 

W*  Ur"'‘  ’  '  i5-'™  Standard 

Flights  .  5  *  .  *  1  /  3  4  rieviation** 


Variables 


crr-(F) 

-13.9 

-0.0 

■0  ( /, 

-33.7 

-lo.h 

-50.8 

18.7 

ccr(sp) 

’  16.0 

4.1  7.1 

6.° 

9,4 

-5.0 

h.O 

Siooe,  bF0 

-1.2990 

—  la 

-1.2073 

-1.2095 

-1 .  WO 

-1.2980 

■  1.0938 

Rag.  M.  Sq. 

3900 

M'.n 

11300 

■  170') 

5500 

6900 

3100 

ft 

1  ".aaad  on 

•veror'4 

•r  nix  Laos 

within  each 

f1 Inht. 

This  otani'cri  deviation  la  derived  from  the  flight  mean  equate  In  the 
analysis  of  variance,  t.e.,  YrVfgfTf^Van’ 9q'uari7’>iT.  The  divisor  six 
arises  from  the  swinging  over  six  legs  within  each  i  light. 
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regression  line  has  never  been  presented  (approximations  could  be  ob¬ 
tained,  perhaps).  In  a  replicated  experiment,  this  lack  of  adequate 
sampling  theory  does  not  create  an  impasse.  Independent  estimates  ob¬ 
tained  fron  repeated  flights  will  permit  direct  estimation  of  the 
variability  of  the  orthogonal  regression  slopes. 

Along  with  computation  of  the  orthogonal  regression  line  we  also 

present  the  regression  mean  square  for  deviations  from  the  regression 

line.  The  magnitude  of  this  mean  square  indicates  the  scatter  of  the 

X  ,  Y  points  about  the  fitted  line.  The  system  whose  data  we  have 
fc  E 

been  presenting  also  provided  an  estimate  of  its  own  position  which  we 
designate  as  Xg,  Yg.  From  this  data  series  we  calculated  a  CCE(SP)  » 

Cross  Course  Error  of  the  system's  Indicated  position.  Table  6  presents 
average  values  of  these  four  statistics  for  the  six  flights  over  the 
area  of  Figure  1. 

As  averages,  these  numbers  in  Table  6  speak  for  themselves.  With 
respect  to  cross  course  error,  if  the  system  actually  was  on  the  left 
side  of  the  programmed  path,  the  deviation  was  designated  as  negative. 
Thus,  we  see  that  the  system  generally  directed  the  flight  slightly 
to  the  left  of  the  programmed  path.  On  the  other  hand,  the  system's 
indication  of  its  position  on  the  average  was  an  even  smaller  deviation 
but  to  the  right  of  the  programmed  path.  For  reference,  the  slope  of 
the  parallel  lines  comprising  the  programmed  path  waa  -1.3032.  Thus, 
the  average  slopes  shown  in  Table  6  agree  quite  well  with  the  desired 
direction.  \ 

Overall  averages,  however,  do  not  tell  the  whole  story.  Hence, 
we  present  average  values  for  the  six  Individual  flights  for  these  same 
variables  in  Table  7.  The  right  hand  column  in  Table  7.  shows  the  standard 
deviations  of  these  averages  as  obtained  from  the  analyses  of  variance 
for  these  four  variables.  Again,  these  data  need  little  explanation. 

We  note  that  for  Flight  No.  4  the  average  of  the  systems'  indicated 
position  also  was  to  the  left  of  the  programmed  path.  Furthermore,  the 
largest  CCE(E)  was  observed  for  this  flight.  We  have  discussed  the 
estimation  of  sampling  error  for  the  regression  orthogonal  slope.  Here 
we  see  the  application  of  this  estimation  even  though  we  have  no  direct 
sampling  theory  for  bEQ.  The  values  shown  for  each  flight  are  based  on 

average  slopes  for  all  six  LEGs.  The  estimated  pooled  standard  deviation 
for  these  slopes  is  only  0.0038. 

SUMMARY.  In  this  paper  we  have  considered  the  assessment  of  the 
performance  of  navigation  and  positioning  systems.  Such  assessment 
comprises  two  parts:  (1)  the  development  of  a  comprehensive  TEST  PLAN; 
and,  (2)  adequate  statistical  analyses  of  the  data  collected.  We  em¬ 
phasize  that  no  more  information  can  be  extracted  from  data  than  has 
been  built  into  the  structure  of  a  test  program  [15].  This  structure 
is  created  by  the  TEST  PLAN. 


-  _ _ _ _ ►»,=  tbqt  pi  4N  ha«  s*,»«  developed  from  an  Outline 

of  the  "General  Features  of  a  Test  Program.  Among  the  ten  items  in¬ 
cluded  in  this  outline  ve  have  directed  particular  attention  to  the 
following: 

(1)  Selection  of  the  system  response  or  performance  measures. 

(2)  Definition  of  an  experimental  unit. 

(3)  Selection  of  the  treatment  combinations. 

(4)  Determination  of  the  pattern  of  experimentation  or  choice 
of  experimental  design. 

(5)  Blocking  of  the  test  program  against  time  or  other  sources 
of  variation  in  the  test  program,  and, 

(6)  The  sample  size  or  how  many  experimental  units  should  be 
completed. 

For  analysis  of  the  test,  data  we  have  considered  the  assessment 
of  both  precision  and  Accuracy.  There  are  many  ways  of  presenting  data 
summaries  to  provide  information  on  both  of  these  characteristics  of 
system  performance.  We  have  illustrated  the  application  of  the  analysis 
of  variance  in  different  ways.  Generally,  we  prefer  this  approach  be¬ 
cause  of  the  ability  to  subdivide  the  total  experimental  variation  into 
sources  associated  with  the  structure  of  the  TEST  FUN.  In  using  some 
results  obtained  from  the  flight  test  program  for  a  navigation  system 
we  have  been  able  to  give  only  a  small  sample  of  the  many  analyses 
performed  for  measuring  both  precision  and  accuracy.  The  latter  we 
note  also  has  been  referred  to  ass  (1)  predictability;  and,  (2) 
reproducibility.  For  one  measure  of  predictability,  the  slope  or 
direction  of  a  flight  path,  we  showed  how  to  measure  directly  the 
variability  of  the  slope  estimates. 
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A  UNIFIED  PROCEDURE  FOR  SELECTING  ALTERNATE  EXPERIMENTAL  DESIGNS 


Edwin  M.  Bartee 
Vanderbilt  University 
Nashville,  Tennessee 

Considerable  knowledge  lias  been  developed  in  the  literature*  that  provides 
for  the  more  effective  design  of  experiments  using,  primarily,  certain  statisti¬ 
cal  techniques  for  analysis  purposes.  Such  methods  are  concerned,  for  the  most, 
part#  with  analysing  the  degroo  of  dopendonce  botweon  the  variablos.  These 
techniques  have  exerted  a  significant  influence  upon  the  amount  of  precision 
and  aoouracy  that  is  realised  in  many  experimante. 

Additional  impact  on  the  optimisation  of  experiments  is  potentially  possible 
through  the  application  of  modeling  teehniques  in  the  synthesis  of  experiments. 
Such  techniques  art  conoemod  with  the  design  of  the  experimental  nodol.  pro¬ 
viding  a  basis  for  systematio  optimisation  of  the  ch  sign  criteria . 

Desisn  Criteria 

As  in  any  enginsoring  design  problem,  the  ultimate  character  of  tho  final 
design  Is  dlotatsd  by  oortsin  design  orlteria.  Some  typical  oriteria  for  an  ex- 
perinsntol  design  are  as  follows i 

1 ,  The  nnmbor  of  faotors  to  be  varied 

2,  Tho  number  of  lovols  to  be  measured  for  caeh  factor 

(a)  Are  levels  qualitative  or  quantitative 

(b)  Are  nonlinear  effect#  to  bs  measured! 

(o)  Are  deviations  to  be  meaeurod  from  a  nominal! 

(d)  Are  all  factori  to  bo  eet  at  an  equal  number  of  levels! 

3,  The  number  of  measurements  of  the  response  variable  to  bs  taken 

(a)  Are  interactions  to  be  measured! 

(b)  Are  there  any  physical  limitations  on  the  number  of  measurements 
in  the  experiment! 

(o)  What  preeieion  la  required  for  measuring  experimental  error! 

This  article  he*  been  reproduced  photographically  from  the  author's 
manuscript. 
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The  synthesis  of  sn  experimental  model  will  be  disoussed  in  ‘three  steps t 


1 .  iseoign  ox  tne  ax.rucu»r<tx  nwuax 

2.  Design  of  the  function*!  model 

3.  Design  of  the  experiment*!  model 

i 

The  first  two  orlteri*  ere  important  In  the  determination  of  the  structural 
model.  Criteria  3  (e)  end  3  (b)  ere  important  in  the  design  of  the  functional 
model.  Criterion  3  (e)  is  the  major  consideration  in  the  design  of  the  experi¬ 
mental  model.  The  ultimate  experimental  modal  is  the  objective  of  the  design 
prooess  disoussed  in  this  paper.  Alternate  standard  experimental  designs  are 
compared  to  the  developed  experimental  model  so  that  a  design  ohoioa  oan  be  made 

that  will  optimise  oonplianoe  with  design  criteria. 

Such  an  optimization  effort  differs  with  the  traditional,  type  in  statistical 
design  of  experiments.  This  traditional  optimization  process  Is  typically  con¬ 


cerned  with  a  trade-off!  between  the  cost  of  experimentation  end  the  statistical 

I  t 

decision.  Such  optimization  would  provide  the  design  criterion  in  3  (c) ;  l.s.» 

I 

a  determination  of  the  number  of  measurements  required  to  provide  a  certain 

l 

precision  in  estimating  experimental  error  so  that  certain  risk  and/or  cost 
requirements  can  be  met.,  Optimization  of  the  experimental  design  in  this  paper 
is  concerned  with  the  selection  of  the  design  that  will  beat  meet  the  design 
criteria  established  for  the  experiment.  One  of  these  design  criteria  usually 
consists  of  the  number  of  measurements  to  be  made  in  order  to  optimize  certain 

statistical  and  cost  requirements. 

The  Structural  Model 

The  etruetural  model  of  an  experiment  is  desoribed  by 


II  ■  k. 

•  1 


H  '  k3  *  ‘  *  km  *  *  *  kp 


number  of  oells  (defined  below)  in  the  experiment 
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k  •  number  of  levels  for  a  factor  or  Independent  variable 


$  8  1  *  2,  3#  « •  •  i  k 

s  *  the  factor  or  independent  variable;  1,  2 ,  p 

p  «*  the  total  number  of  factors  in  an  experiment 


The  simplest  form  of  an  experiment  is  the  case  of  one  factor,  for  example 
X| ,  at  one  level,  so  that 

p  »  1,  k1  “  1 

and  thus,  from  Eq.  (1),  the  structural  model  becomes 

Ns  *  k1  "  1 

This  model  is  called  a  cell,  the  basio  structural  unit  of  an  experiment.  The 
next  form  of  an  experiment  is  the  oase  of  one  factor  at  tv/o  or  more  lovols,  so 
that.  p  •  1,  ■  2,  3.  ...,  It. 

“  .  I 

and  thus,  the  struotural  model  becomes 

Hg  *  2,  3,  knells 

The  next  form  or  level  of  an  experiment  is  illustrated  by  a  oase  in 
which  there  are  two  factors  at  two  or  more  levels.  Thus 

,p  *  2,  ■  2,  3.  ...i  kj,  "  2*  ^  **,lc2 

and  the  structural  model  beoomes  , 

H#  *  •  k2  cells 

Consider  another  example.  A  three-factor  experiment  is  described  by 
P  ■  3.  kj  *  2,  kg  *  3i  kj  «*  3 

and 


»s  ■  k^  •  k2  ■  •  2*3*3  *  16  cells 
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A  special  case  of  the  structural  model  occurs  when  the  experiment  is 
symmetrical,  meaning  that  all  factors  have  an  equal  number  of  levels.  There¬ 
fore,  when 

kj  ■  k2  ■  kj  ■  •••  ■  »  k^ 

■q.  (1)  becomes 

Ks  “  k1  '  k2  *  ’  *  “m  ’  ’  *  kP  “  ^ 

To  illustrate,  let  us  consider  an  experiment  with  two  faotors,  each  at 


two  levels,  described  by 

p  »  2,  ■  k2  ■  k  ■  2 


Thus, 


UB  ■  k^  «•  2^  ■  4  oells 


For  another  example  consider  a  case  of  the  symmetrical  model  with  three  factor, 
eaoh  at  two  levels. 


P  •  3.  k  -  2 


■  23  -  8 

Thus,  Eq.  (2)  determines  the  number  of  cells  for  any  symmetrical  model  with  p 
factors  eaoh,  at  an  equal  number  of  k  levels. 

The  design  oriteria  that  are  described  by  the  structural  model  ares 

1.  The  number  of  factors  ■> 

2.  The  number  of  lovels  per  faotor 

These  criteria  are  determined  by  the  objectives  of  the  experiment,  the 

measurability  of  the  faotors,  the  interest  in  nonlinear  effeots,  etc.  They 

should  not  be  dictated  by  any  limitations  upon  the  total  number  of  measurements 

that  oan  be  made  of  the  response  variable.  Suoh  limitations,  or  lack  of  them, 

’  / 
is  the  eonoern  of  the  functional  model. 


The  Functional  Model 

The  functional  model  determines  how  many  cells  in  the  structural  model  will 
contain  a  response  measurement.  Such  functional  models  are  either  comoloto  or 
Incomplete.  A  functional  model  is  considered  to  be  complete  when  all  cells 

contain  a  response.  A  functional  model  is  incomplete  when  the  number  of 

/  , 

responses  are  systematically  limited,  so  that  the  number  of)  responses  is  less 
than  the  number  of  cells.  Each  of  theso  basio  types  of  functional  modols  will 
now  be  discussed. 

The  necessary  and  sufficient  conditions  for  a  complete  functional  model 

are: 

Mf  "  Ns  "  k1  *  k2  *  *  *  km  *  *  *  kp  (3) 

where: 

Na  *  the  number  of  cells'  in  the  experiment. 

Nf  «  the  number  of  responses  in  the  experiment, 
k  ■  the  number jot  faotor  levels  h  2, 
p  ■  the  total  number  of  factors  *  1 . 

1  . '  ; 

*  ■  If  2*  3,  .f.,  p. 

For  the  special  case  of  symmetry  where 

k1  "  k2  “  k3  "  •  •  •  ■  kp 

'  ■  \  j 

Equation  (3)  oan  be  written  as  / 

Hf  «  Ng  «  k>  (4) 

In  both  Bq.  (3)  and  (4),  it  can  be  observed  that  the  npmber  of  cells  in  the 
structural  model  (Ng)  and  the  number  of  responses  in  the  functional  model  (Nf) 
are  equal.  This  equality  is  the  basio  oharactoristia  of  a  complete  model.  In 
other  words,  for  every  oell  there  is  a  response,  or 


t 


for  example,  given  the  experiment  with  two  factors,  Xj  and  x2  ,  one  at  two 
levels  and  the  other  *t  three  levels,  we  have 

P  ■  2,  k,  ■  2,  k2  -  3 

*  -  Mf  -  kt  •  k2  -  2  •  3  -  6 

A  functional  model  is  incomplete  when 

»t  <  ». 

or  when  the  number  of  responses  in’  the  experimental  model  are  determined,  in 
some  systematic  manner,  to  be  less  than  the  number  of  cells.  Our  concern  at 

'  i. 

this  point  is  to  consider  the  fundamental  methods  that  are  involved  in  designing 
such  an  incomplete  model. 

FUnotional  models  ean  be  made  Incomplete  in  three  fundamental  ways,  Ijhe 
first  of  these  is  ths  restriction  of  responses  exponentially,  so  that  the 
number  of  excluded  responsos  are  determined  by  restriction  with  the  factors  in 
the  model.  The  second  method  for  designing  incomplete  models  is  to  restrict  ; 
ths  responses  linearly,  so  that  ths  number  of  exoludsd  responses  in  a  model  are 
determined  by  restriction  with  a  certain  number  of  levels  of  a  single  faotor 
in  the  model.  The  third  method  consists  of  a  combination  of  the  firet  two,  in 
which  ease  th-  restriction  of  responses  is  acoompliohed  by  both  exponential 
and  linear  methods.  Each  of  these  methods  will  now  be  discussed,  - 

i 

i 

i 
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From  Eq.  ii).  the  necessary  and  sufficient  conditions  for  an  incomplete 
functional  model  whose  responses  are  restricted  with  factors  are 

K  “  1  2  m_  p  (5) 

*  kz  *  *  *  *1  *  •  *  kq 

where t  q  ■  the  number  of  factors  restricting  the  number 

C 

of  responses  In  the  model.  1  ■  0,  1,  2,  ...  q 
(Non-negative  integers.) 
m  B  1|  2|  *»«t  P 

When  q  is  equal  to  tero,  no  restriction  on  responses  exists.  Consider  the 
oase  of  s  structural  model  with  three  faotoro,  ,  x^,  and  Xy  with 

N#  -  k,  •  k2  •  k3  »  2  •  4  •  2  -  16 
in  which  the  number  of  responses  is  to  be  restricted  by  one  factor,  for  example 
Xa.  Therefore,  we  have  one  restricting  factor,  making 

. <i  -  i ; 

and,  thus  from  Eq.  (5)  ^ 

Nf  A  k1  ’  k2  ‘  kV  -  il til  - 
)  H  5 

giving  four  responses  that  are  contained  in  the  sixteen  cells  of  the  structural 
model. 

Consider  another  oase.  Suppose  that  a  structural  model  contained  four 
factors  Xj ,  x2,  Xy  x^,  with 

Hs  “  k1  *  k2  ‘  k3  ‘  k4  “  4  *  *  ’  4  *  4  B  320 
Suppose  that  the  number  of  responses  in  the  functional  model  is  to  be  restricted 
by  the  two  factors,  x2  and  Xy  Thus,  we  have  the  1th  factors  (1  ■  2,  3) 
restricting,  so  that 
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»>-fi  .  3 

3q 

3**  •  ~  »  9 

q  *  2 


The  second  method  for  restricting  the  responsos  in  an  incomplete  functional 
model  limits  the  responses  within  the  levels  of  a  particular  factor  rather  than 
with  q  number  of  factors.  This  is  done  by  subtracting  the  total  number  of 
blank  cells  for  a  particular  factor  from  the  total  number  of  cells  in  the 
structural  nodal.  Thus,  from  Bq.  (3),  ve  have 


Nf  ”  k1  *  k2  *  *  *  km  *  *  *  kp  *(  cmkn  (7> 

where t  c  ■  the  number  of  blank  cells  in  each  level  of  the 

A 

mth  faotor.  (a  non-negative  integer) 

at. 

■  the  number  of  levels  of  the  nr  factor. 


The  m^1  factor  can  be  any  ;^>ne  of  the  p  factors  in  the  nodol.  For  example,  a 
p  «  2  model  oan  bo  systematically  limited  by  arbitrarily  determining  the  number 
of  blank  cells  to  exist  in "each  level  of  one  of  tbo  two  factors,  Xj  and  x2»  This 
is  e.  The  number  of  responses,  M^,  is  then  oaleulated  from  Eq.  (7),  Consider 
an  example  in  which  the  levels  for  the  first  factor  are  six,  kj  ■  6,  and  the 
levels  for  the  second  factor  are  three,  k2  ■  3*  If  we  choose  to  restrict  the 
first  factor,  Xj,  so  that  each  factor  level  has  one  blank  cell,  then, 

*1  •  6,  h2  -  3,  *  -  1,  *  1 

The  number  of  cells  are 
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*a  *  k1  *  k2  “  6  *  ^  «  « 
irnd  the  number  of  responses  are,  from  Eq,  (7) 

Hf  »  kt  •  k2  -  Cjkj  -  k1  (k2  -  Cj)  ■  6  (3  -  1)  *»  12 

In  the  ease  of  a  symmetrical  model,  we  determine  the  incomplete  functional 
model  from  Eq.  (7)  to  be 


Nf  “  k1  *  k2  **•  "* kp  '  VSa 

and  sincer 

k1  -  k2  "  “a  "  kp 

- 

*f  ■  kp  -  ckp_1  (8) 

where: 

e  «*  the  number  of  blank  cells  in  the  level  of  any 

p  factor. 

Bq.  (8)  gives  emphasis  to  the  linear  feature  of  this  method.  In  the  case 
of  the  model 


N#  «  k1*  ■  34  -  81 

we  could  limit  the  number  of  responses  by  creating  blank  cells  in  the  factors. 
For  example,  with  o  ■  1 ,  we  can  calculate  from  8q.  (8)  ■. 

Vf  -  kp  -  okP"1  ! 

•  34  -  <1)  3<K*1  -  81-27  »  54 

The  model  can  be  used  in  a  different,  and  more  useful,  way  from  a  design  stand¬ 
point.  As  an  example,  what  value  of  0  is  required  to  reduce  the  model 

«  kp  -  55  -  3123 

to  the  functional  model  of 

Mf  -  625 

This  is  determined  from  Bq.  (8)  thus: 

Hf  -  kp  -  ckp-1  -  625 

5^  -  c5^  e.  625 

\ 
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C  =  312 5  •  625  *  4 

Therefore,  the  functional  nodal  can  be  restricted  to  625  responses  by  providing 
for  four  blank  cells  in  each  factor  level. 

In  oidor  to  fhrther  increase  the  possible  combinations  of  N^  values,  the 
third  nethod  utilises  both  the  q  and  c  criteria.  This  oan  be  accomplished  by 

I 

Bq.  (6)  and  Bq.  (8)  to  become 

.  kP-O  -  t  lMJ  «) 

so  that  the  number  of  restricting  factors,  q«  and  the  number  of  blank  colls 
per  factor  level,  o,  can  be  used  to  determine  a  particular  number  of  responses 
for  a  given  model.  The  application  of  Eq.  ($)  will  be  illustrated  by  an 
example.  Suppose  that  it  is  desirable  to  restrict  the  number  of  responses  for 
tho  model 

11^  ■  «  4^  ■  64 

to  eight  responses.  This  can  be  done  by  using  Eq,  (9),  and  following  a 
systematic  procedure.  First,  assume  c  ■  0,  and  q  •  1 

«f  -  lf»  -  ck^'1 

■  *3-!  .  „  o  ■  1$ 

whioh  is  greater  than  the  desired  number.  Beat,  keep  c  ■  0  and  assume  q  ■  2 

llf  ■  4^*2  -  0  ■  4 

which  is  less  than  the  desired  number.  Therefore,  hold  q  ■  1,  and  assume  o  “  1 

g  *  I*?-1  -  43-1*1  -16-4-12 

f 

which  is  more  than  desired.  Next,  hold  q  *  1  and  set  e  ■  2 

Uf  .  43”1  -  (2)  45-1"1  -16-8-8 

which  is  the  desired  number  of  responses. 
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Suppose  that  we  wanted  to  determine  how  to  design  a  functional  model  with 
nine  responses  for  the  model 

\  ■  &  -  36  ■  729 

with  k  •  3,  the  desired  number  of  responses  is  k2  ■  9.  It  oan  be  seen  that  snob 
a  value  for  Hf  Is  possible  in  two  ways.  First,  Nf  ■  k2  for  the  ease  when 

q*p-3*6-3B3 

o»k-1»3«l»2 

Therefore,  in  this  problem 

<1  ■  3,  c*2 

»f  ■  k™  -  o  kP-*-1 

-  3s*3  -  <2)  36"3"1 

•  2?  -  18  *  9 

The  sane  number  of  responses  can  be  obtained  with  a  different  combination:  of 
q  and  e.  «  k2  is  possible  with 

i 

q  »  p  -  2  *  6-2  *  4 
0*0 

Therefore,  the  node!  becomes 

*f  -  kP^  -  36J*  -  32  -  9 

A  eooplote  functional  model  is  tbs  sans  as  a  factorial  experiment,  with  a 
single  response  in  eaoh  cell.  An  incomplete  functional  model  is  desirable  when 
there  is  no  interest  in  interaction  effects  and  the  total  number  of  measurements 
required  is  leas  than  Ng.  An  incooplsts  model  is  necessary  when  the  total  possible 
number  of  measurements  is  less  than  Ng.  More  specifically,  the  design  of  the 
functional  model  is  made  to  meet  the  following  design  criteria: 
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"trO  or  IcOa 


than  N0. 

» 

2.  The  total  number  of  measurements  is  limited  to  some  number  less  than 

N  because  of  some  physical  limitation  of  the  experimental  situation 
ft 

or  equipment. 

Jhs.  ifo>P£3jiei&&  Ms! 

The  final  step  in  the  .qynthesls  of  an  experiment  is  to  design  tho  experimen¬ 
tal  model.  The  experimental  model  is  described  by 


M  «*  n  Nj 


(to) 


where:  ’ 

n  ■  the  number  of  replications  of  the  experiment 
Nf  »  the  number  of  responses  in  the  functional  model 
N  *  the  total  number  of  responses  in  the  experiment 
From  Eq.  (9)  and  Eq.(10)  we.  got 

N  ■  n  Nf  «  n  (kp-q  -  c  kH’b  (11) 

which  provides  a  general  expression' for  a  symmetrical  experimental  model. 

Bq.  (14)  thus  defines  the  experimental  model  as  follows i 

The  total  number  of  responses  in  an  experiment  is  a  function 
of  the  number  of  faotore  (p) ,  the  number  of  faotor  levels  (k),  tlie'j 
number  of  faotor  restrictions  (q)t  the  number  of  eell  restrictions/ 

(e),  and  the  number  of  replications  (n). 

Oiven  the  number  of  factors  and  faotor  levels,  the  number  of  possible 
values  for  N  can  be  determined  by  certain  combinations  of  values  for  n,  q, 
and  o.  For  example,  if  it  is  desirable  to  design  an  experimental  modol  with 
54  responses  of  the  type 
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II  B  1,P  m  —  Si 

S'" 

vs  can  set  q  ■  0,  c  »  1,  and  n  »  1  and  get 

N  -  n  (kP^  -  o  kP-*-1) 

-  1  34-0  -  (1)  34^"1 
*  81  -  2?  ■  5^ 

Table  1  provides  a  general  tabulation  of  the  experimental  model  in  Bq.  (11) 


TABLE  1 .  Values  of  N  for  All  Values  of  q»  c,  and  n  In  a 


Symmetrical  Experimental  Model 


*  -  - 

0 

* 

k 

k  -  1 

k  -  2 

lUMB 

0 

1 

p 

0 

n/k 

2  n/k 

3  n/k  ... 

n 

p  «  1 

0 

n 

2  n 

3n  .  .  . 

nk 

p  -  2 

0 

nk 

2  nk 

3  nk  ... 

nk2 

P  -  3 

0 

nk2 

2  nk2 

3  nk2  ... 

nk? 

• 

e 

e 

• 

•  * 

• 

• 

• 

■r-  *-•(  c 

• 

.  • 

• 

s 

• 

V  • 

• 

t 

•  . 

0 

0 

2  nkP"1 

•i 

3  nkP”1  .  .  . 

nkP 

The  number  of  responses,  N,  for  a  symmetrical  experimental  model  oan  be 
determined  If  given  the  values  for  p,  k,  q,  o,  and  n.  As  an 
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Avamnia  nf  A+.«  »•«*.  mmnnsB  that  w«  have  a  nodel  with  d  =  5  factors  and  each 

..g.—  m  *  * 


factor  has  k  =  5  levels  so  that 

V  “  55  3  3,125 

Assume  that  the  experimental  nodel  is  to  contain  forty-five  responses.  The 


responses  are  first  limited  by 

qap-2*5-2"3  factors 


Also,  the  responses  are  further  restricted  by 

0«.k-3«5-3*»2  blank  cells  per  factor 
When  q  “  p  -  2  and  c  =  k  -  3 

N  -  3  nk 

and  with  k  ■  5 

N  ■  3  «(5)  -  15  It  - 
n  ■  3 

Selection  of  Optimal  Alternate  Designs 

the  experimental  model  providos  the  specifications  neoessary  for  the  final 
experimental  design  to  meat  the  established  design  criteria,  as  to  total  number 

v.  '* 

•of  responses.  Such  a  selection  is  not  concerted  with  the  problems  of  balancing 
the  response#  in  the  eells  or  randomising  the  arrangement  of  the  responses.  These 
are,  considerations  made  in  eertaln  standard  designs  with  which  the  subject 
design  procedure  is  not  conoembd . 

The  sjpithssis  of  any  experiment  can  be  deaorlbed  by  its  experimental  > 

i 

model.  For  example,  a  complete  factorial  experiment  is  described  by  the  follow¬ 
ing  neoessary  and  sufficient  conditions i 

n  ■  1,  2,  3,  •  •  • ,  k  ■  2,  3,  ....  p  ®  2,  3,  q  *)0,  c  *  0 
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A  casa  ftf  mw*h  a  fifttflHal  nvnAHoumfe  An 

•  « 

n  ■  1 ,  p  ■  3»  k  »  4,  q  ■  0,  c  ■  0 
and  from  Bq.  (11) 

II  -  n  (kP^  -  e  k*5^-1)  -  (1)  (4*)  -  64 
examples  „of  other  nodola  for  certain  traditional  experiments  aro  listed  as 
followst 

1 .  A  fim-jai  classification  experiment  with  five  responses  in  each  of 

four  columns  of  a  single  faetdr  is  described  by 
n*5.  p*1.  k  ■  4,  q  a  0,  c  *  0 

N  »  n  (kP^  -  o  kP^"1) 

-  (5)  (41-0)  -  20 

Suoh  a  modol  thus  explains  the  one-way  classification  experiment  as  a 
single  factor  experiment  that  la  replioated. 

2.  Consider  a  nested  experiment  with  three  factors)  x,,  with  two  levels, 

%2  i  with  four  levels,  and  x^  with  two0 levels.  Factor  Xg  is  suoh 
that  only  half  of  its  levels  are  crossed  with  each  of  the  two  levels 
of  Xj.  Thusi  ! 

P  ■  3.  kt  -  2,  k2  ■  4,  kj'  -  2 

Fron  Sq.  (1)  1 

N,  •  k,  •  k2  •  kj  -  2  x  4  x  2  •  1$ 

Since  factor  Xj  restricts  the  number  of  responses  in  the  experiment 

q  «  1 ,  kA  ■  kj  ■  2 

and,  thus,  from  Sq.  (5) 

*  k1  *  k2  *  k3  -  2  ■*  U  •  2  m  8 
k,  2— 

with  only  one  replicate 

N  -  n  Nf  -  (1)(3)  -  8 
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A  hierarchical  layout  becomes 


A  matrix  layout  becomes 


?•  Consider  a  latlnSquare  experiment.  The  necessary  and  sufficient 
conditions  for  tilis  symmetrical  restricted  model  are 

p  ■  3,  q  "  p  -  2,  k  j*  p  f  1,  *  o  ■  0 
The  minimum  case  ooours  when  there  are  three  faotors,  each  at  two' 
levels,  with 

P  ■  3»  k  *  2,  q  ■  p  -  2  ■  1 
and  the  number  of  cells  and  responses  are 

Ng  ■  kp  »  2^  ■  8 

Nf  ■  kp-<*  ■  7?  m  4 

so  the  experiment  contains  a  total  of  four  responses  In  eight  cells. 
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in 

■  I 

B 

4.  A  Graeco-Latin  Square  experiment  is  described  by  the  following 

•) 

necessary  and  sufficient  conditions 

e  «  0,  k  -  3*  p  =  4,  q  *  p  -  2  *  2 

For  the  case  in  which  k  *  3,  the  number  of  cells  in  the  structural 

model  would  be 

Ha  *  kp  *  34  =  81 
and  the  number  of  responses  would  be 

Nf  =  V?*  =  y*“2  =  9 

Only  one  replicate  is  taken.  Thus, 

II  «  n  Nf  =  (1)  (9)  =  9 

5.  An  incomplete  block  experiment  is  represented  by  the  incomplete 
functional  model  whose  necessary  and  sufficient  conditions  are 
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■  k<  *  k2  ■  6  •  3  *  18  cells 
n  *  1 

N  -  k1  •  k2  -  c2k2  «  6  *  3  -  2(3)  *  12 

so  the  twelve  responses  are  to  be  balanced  In  the  eighteen  cells  of 
the  incomplete  block  design. 

6.  A  symnetrioal  Incomplete  block  experiment  is  described  by  the  necessary 
and  sufficient  conditions  from  Eq.  (8) 

k  4  3,  p  «  2,  c  ■  1,  2,  k  -  1 

n  *  1 

N  »  k2  -  ck 

vhere:  Cne  of  the  two  faotor-  is  a  block. 

7.  A  Youden  Square  experiment  is  described  by  the  following  necessary 
and  sufficient  conditions  from  Eq.  (9) 
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N  =  n  (kP^  -  c 

k  -  2,  p  =»  3,  q  ■  1,  c  «  1.  2,  •••,  k  -  1 

n  *  1 

A  specific  example  is  a  case  in  which 

p  «  3,  k  «  4,  q  B  t,  ca1,  n  ■  1 

N  ■  kp  ■  4^  ■  64 

8 

N  -  k*^  -  c  kP*^’1 

a  k^  <1  -  f)  -  43"1  <1  -  J)  -  12 

8.  A  lattice  square  experiment  is  described  as  a  type  of  incomplete 
block  (See  6  above)  that  is  replicated.  Thus 
N  ■  n  (kt  •  k2  -  c2k2) 

where:  2  <  n  *  t  +  la  t  4  3 

k.}  a  n  t  (blocks) 

k2  *  t  (treatments) 

«2  "  k1  *  1 

Example: 

t  ■  3,  n  ■  4,  k.j  ■  12,  k2  »  9,  ez  a  ^ 

K#  ■  12  •  9  *  108 

N  ■  4  (12  •  9  -  11  •  9)  •  3 6 

Another  example  of  a  lattloe  square  will  demonstrate  the  relationship 
between  the  structural,  functional,  and  experimental  model  more  clearly. 

A  13  x  13  balanced  lattice  square  will  be  used  to  illustrate,  (t  *  13,  n  "7.) 

Structurally  speaking,  the  experiment  eonsists  of  two  factors: 
blocks  at  kj  =  21  levels  and  treatments  at  k2  ■  169  levels.  Thus 

HS  «  ki  •  k2  «=  21  •  169  -  3,549  cells 
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_  -  _ _  «-  a rvi  its  characteristic  feature  is  thit 

1  tie  iaitwwivii<a«  *■  —  — - r  - 

only  one  treatment  can  occur  in  each  replicate.  Thus,  the  functional 
■odel  is  restricted  so  that  for  each  treatment  all  ceUs  are  empty  except 

one.  Thus  c2  *  20,  giving 

Hf  «  k1  ‘  k2  -  Cgkg  =  3 .  5**9  -  20(169)  =  169 

The  experiment  is  replicated  7  times, thus  the  experimental  model 
becomes 

N  «  nNf  -  7(169)  »  1,183  responses 
which  is  the  total  number  of  responses  to  be  balanced. 

9.  Following  are  a  number  of  miscellaneous  jncicmB3te^  felfigk  designs 
with  their  corresponding  structural,  functional,  and  experimental 

models. 

BALANCED  DESIQN  FOR  9  TREATMENTS  IN  BLOCKS  OF  3  UNITS 


Block 

JBgE&. 

JL 

BsSjJI 

Ren.  HI 

Ren. 

IV 

<0 

1  2 

3 

(4) 

14  7  (7) 

i.i-2 

(10) 

1  8 

6 

(2) 

Li. 

_6 

(5) 

298  (8) 

7  2  6 

(ID 

4  2 

—2 

(3) 

2L-S- 

3 

(6) 

369  (9) 

4-1-1 

(12) 

2-1 

-2 

p 

-  2. 

V  ■  9, 

k2  -  12 

-  kt 

♦  k2  «  9  •  12 

-  108 

ci 

«  11 

"f 

-  >. 

-  Cjkj  ■  108  -  (11)  9  * 

9 

n  «  4 

N  «  n  Nf  -  4  (9)  “  36 


393 


BALANCED  D:-I;I  »*.  POfi  7  TREATl'iEMS  IN  BLOCKS  OF  3  UNITS 

Block 


(1) 

1  2  4 

(3) 

_4  6  (5)  J_ 

j _ 6 

(7)  1 

/ 

l -J,-  2 

(4) 

4 

^  7  \b)  2_ 

JL-Z 

P  » 

2, 

k  a  7 

- 

kP 

S 

*o 

fo 

II 

£ 

0  «= 

4 

Nf  - 

N« 

- 

c  k**”1  a  49  - 

(4)  ? 

a  21 

n  a 

1 

N  > 

n  Nf 

a 

It 

oj 

BALANCED  DESIGN  FOR  9  TREATMENTS  IN  4  LATTICE  SQUARES 


Rep.  I 

Rep.  II 

Rep.  HI 

Rep.  IV 

Columns 

Rows 

urn  3) 

(IBfcKfi) 

(2LMJ2) 

OftiOt)M2) 

<t> 

(4)  14  7 

(7)  188 

00)  1-  9-  S 

(2) 

4  5  6 

(5)  2  5  8 

(8)  9  2  4 

01)  _6_  2  7 

(3) 

7  8  9 

(6)  i_  6  9 

(9)  JLJL-2 

(12)  8  4  3 

P  * 

3.  ki  ■  9, 

k2  “ 

12,  kj  a  12 

Ns  " 

k1  ’  k2  *  k3  ■ 

9  •  12 

•  12  -  1,296 

C1  “ 

143 

Nf  " 

Ns  -  clkl  e 

1,296  - 

143  (9)  »  9 

n  “  4 

N  «  n  »f  »  4  (9)  .  36 


394 


BALANCED  DESIGN  FOR  ?  TREATMENTS  IN  AN  INCOMPLETE  LATIN  SQUARE 

.1 _ _ _ /  m _ i._\ 

WVAUVUIO  \  iUX.  VV  t\0  / 

(D  (2)  (3)  (4)  (5)  (6)  (7) 

Row  3 

(1)  1  234567 

(2)  2  3  4  5  6  ?  1 

(3)  4  5  6  7  1  2  3 

P  -  3.  k  ■  7 

n  *  kp  »  73  “  343 
q  -  1,  c  ««  4 

Nf  -  k1^  -  e  k^’1  -  73"1  -  (4)  7>U1  *  21 
n  «  1 

N  -  n  ■  21 

10.  A  composite  design  is  used  to  estimate  the  regression  coefficients  for 
s  second  degree  ploynomlal .  These  designs  are  traditionally  oonatraoted 
by  adding  further  treatment  combinations  to  those  obtained  from  a  iP 
factorial.  Suoh  designs  are  doscribed  here  as  replioated  incomplete 
models  with  N#  •  3P.  Such  an  approach  recognizes  the  necessity  for 
three  factor  levels  to  measure  second  degree  effeots.  Thus 
q  *  p  -  1,  c  -  2 
Hf  -  3P*’P+1  -  (2)  (3>P*P  -3-2-1 
n  -  2**  +  2  p  1  (central  design) 

n  *  2P  +  2  p  +  p  (noncentral  design) 

N  -  nNj  -  2^  +  2p  +  1  (central) 

K,  «  n  Nj  -  2P  +  2p  +  p  (noncentral) 

Examples 
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1 .  Design  Criteria .  A  flight  vehicle  trajectory  is  to  be  designed 
so  that  a  multistage  rocket  nay  place  a  payload  into  a  circular  orbit 
about  the  earth.  An  ('rnniH-i«nt  i"  to  be  designed  to  dctcmino  hvm 

the  first  stage  booster  thrust  program  affects  the  amount  of  mass 
which  is  injected  into  a  circular  orbit  with  an  altitude  of  100  miles. 
The  thrust  program  of  the  booster  consists  of  programmed  adjustments 
in  the  angle  of  attack  for  a  given  time  period.  The  length  of  the  first 
stage  bum,  t^,  is  determined  by  the  propellant  loading  of  the  first 
stage.  The  parameters  that  control  the  thrust  program  are  (1)  the 
Initial  rate  of  increase  of  the  angle  of  attack,  R,  and  (2)  the  length 
of  time  this  rate  is  flown,  tj . 

The  factor,  R,  is  to  be  sot  at  six  different  rates  of  increase 
and  tj  is  set  at  four  time  levels.  The  flight  is  to  be  simulated  on 
a  digital  computer.  Previous  experience  in  similar  studies  indicates 
that  each  run  (response)  requires  about  3/if  of  a  minute  of  computer 
time.  The  computer  "turn  around”  time  is  very  slow.  It  is  necessary 
to  obtain  the  maximum  priority  time  required  on  the  computer,  which  is 
10  minutes.  Therefore,  a  maximum  of  13  runs  can  be  made.  There  is  no 
interest  in  interaction  effects. 

Model  Synthesis.  The  structural  mold,  according  to  ihe  above 
design  criteria,  is  determined  to  be 

p  »  2  factors,  «  6  levels,  k2  »  4  levels 
.  N#  «  k,j  *  k2  «  6*4  a  24  cells 

The  maximum  number  of  responses  requires  an  incomplete  functional 
model  restricted  by  the  o  criteria.  Thus 

Nf  "  k1  *  k2  “  c1k1 
Setting  Nj,  a  13  (maximum)  we  calculate 
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which  is  not  an  integer.  Using  =  12  we  get 

Cj  3  2 

Therefore!  the  number  of  blank  cells  per  level  of  factor  R  18  2, 
providing  a  total  of  11  degrees  of  freedom.  With  the  two  main  ef foots 
requiring  5  and  3  degrees  of  freedom,  respectively,  the  experimental 
error  is  estimated  with  three  degrees  of  freedom  since  the  experiment 
is  not  replicated.  Thus 

n  =  1 

M  ■  n  N{  -  (1)  (12)  =  12 

2.  Design  Criteria.  Two  different  analogue- to-digital  converters 
are  contained  in  test  stations  used  in  checking  out  a  particular 
instrument  unit.  An  experiment  is  designed  to  determine  the  causes  of 
variation  in  the  digital  output  of  these  converters.  The  response 
variable  is  the  difference  between  input  voltage  and  output  voltage. 

The  variables  to  be  measured  are  (1)  input  voltage,  (2)  converter  units, 
and  (3)  adjustments.  The  input  voltage  is  to  be  set  at  two  levels, 

-10  volts  and  +10  volts.  The  number  of  converters  are  limited  to  two. 

The  adjustments  consist  of  gain  and  balance  settings  as  specified  by 
the  manufacturer.  Four  different  adjustments  will  be  node.  The 
ad>istments  are  unique  with  each  unit  and,  therefore,  they  cannot  be 
duplicated  between  the  two  converters.  Thus,  the  first  two  adjustments 
will  be  unique  with  the  first  unit  and  the  second  two  adjustments  will  be 
unique  with  the  second  unit.  All  possible  interactions  are  to  be 
measured .  The  optimal  degrees  of  freedom  for  the  error  estimate,  con¬ 
sidering  cost  of  experimentation  and  desired  decision  confidence  levels, 
has  been  determined  to  be  1 6  in  a  previous  study. 
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Model  Synthesis.  The  structural  model  is 


",  -  k1  '  k2  '  kJ  *  2  •  2  •  *  *  16 


The  convortcr  factor  restricts  the  adjustment  factor  thus  providing 
conditions  for  an  incomplete  functional  model,  restricted  by  the  q 
criteria .  Thus 


c  *s  0,  q  »  1  (converters,  kg  *  2) 

N.  ■  K1  k2  k3  =  2  •  2  •  4  =  8 
k2  2 

The  optimal  degrees  of  freedom  for  the  error  estimate,  1 6,  is  provided 
by  replicating  the  functional  model.  Thus 

n  «  3 

N  «  n!Jf  «  3  (8)  «  24 
The  degrees  of  freedom  are  partitioned  as  follows: 


Source  of  Variation 
Converters  (C) 
Adjustments  (A) 
Voltage  (V) 

C  V 
A  V 
Error 


Degrees  of  Freedom 
1 
2 
t 
1 
2 
16 


Total  23 

A  layout  of  the  selected  experiment, which  is  a  nested  factorial,  is 
shown  in  Table  2, 
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i 

Adjustments 
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Adjustments  ! 

Voltage 
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X 
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X 

X 

X 

+10 

X 

X 

* 

z 

z 

X 

X 

X 

Table  2.  layout 'of  Analogue- to-Digital 
Converter  Experiment 


3.  Design  Criteria.  An  electronic  aanuf-  irer  has  designed  a 
component  board  using  four  capacitors  to  estab7  .<■  a  time  base.  Be 
vishes  to  test  five  different  brands  of  the  capa^tors  in  the 
component  boards.  Four  capacitors  are  placed  in  parallel  and  then 
connected  through  a  resistor  to  an  input  plug  where  a  fixed  voltage 
aajr  be  applied.  The  voltage  across  the  capacitors  is  connected  to  an 
output  Jack.  The  test  is  Bade  by  applying  a  fixed  voltage  to  the  plug 
at  the  input  of  the  component  board.  The  output  Jack  is  monitored  with 
an  oscilloscope  to  Measure  the  tine  required  for  the  output  voltage  to 
rise  to  a  specified  amplitude. 

The  response  variable  (T)  is  the  tine  required  for  the  output 
of  the  conponent  board  to  rise  to  a  specified  amplitude  upon  application 
of  a  fixed  input  voltage.  The  factors  of  interest  are  capacitor 
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brands  (C)  and  component  boards  (B).  Since  both  factors  are  qualitative, 
nonlinear  effects  are  not  applicable.  Also,  past  experience  in  tests  of 
inis  type  has  shown  negligible  interaction  between  the  capacitors 
and  component  boards.  Since  infornution  is  desired  on  the  capacitors 
only,  the  same  resistor  will  be  used  for  each  test.  A.  set  of 
terminals  allows  the  resistor  to  be  plugged  in  or  removod  from  the 
component  board.  Five  different  capacitor  brands  are,  therefore,  . 
to  be  tested  in  a  circuit  that  is  limited  to  four  capacitors.  A 
minimum  of  10  degrees  of  freedom  is  required  to  make  an  error  estimate. 

Model  Synthesis.  Since  five  brands  are  being  tested,  it  would 
seem  reasonable  to  test  these  brands  in  five  different  component 
boards.  We,  therefore,  have  a  symmetrical  model.  The  struotural 
model  is  , 

p  ■  2,  k  «  .5 

,  N#  «  kp  «  52  *  25 

Since  there  are  four  capacitors  in  the  circuit  but  five  different 
brands  we  will  have  one  missing  valuo  in  each  level  of  capacitdr 
brand.  Thus,  the  functional  model  is  incomplete  with 
q  «  0,  k  *  5,  p  «  2,  c  ■  1 
Nf  »  kp  -  c  kp_1  *  52  -  (l)^2”1  “  20 

The  degrees  of  freedom  are 

Source  of  Variation  Degreoa  of  Freedom 

Capacitors  (C)  4 

Boards  (B)  4 

Error  12 

Total  20 

Only  one  replicate  is  required  since  the  minimum  of  10  degrees 

of  freedom  is  met.  The  experimental  model  is 


n  *  1 


N  >  n  Nf  •  (1)  (20)  »  20 
A  balanced  layout  of  the  exporiraent  is  shown  in  Table  3«  as 
an  incomplete  block  design. 
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TABLE  3.  Incomplete  Block  Design  for  Capacitor  Experiment 


Summary 

The  modeling  of  experiments  has  been  described  as  a  three-phase  process, 

4 

namely 

1.  Designing  the  structural  model 

2.  Designing  the  functional  model 

3.  Designing  the  experimental  model 

The  structural  model  determines  the  number  of  cells  in  the  experiment  as 
a  function  of  the  number  of  factors  and  the  levels  for  each  factor.  For  the 
symmetrical  case  the  structural  model  is 

N  **  kp 

9 
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The  functional  model  determines  the  number  of  responses  to  be  taken  in 
the  structural  model.  A  complete  symmetrical  functional  model  is  expressed  as 


A  functional  model  can  be  incomplete  in  three  ways.  First,  if  the  responses 
are  restricted  by  q  number  of  factors,  the  symmetrical  functional  model  becomes 

Nf  =  kpK1 

Second,  if  the  responses  are  restricted  by  c  cells  within  a  factor,  the 
symmetrical  functional  model  becomes 

Nf  =  kp  -  c  kp_1 

*  kP  (1  - 

Third,  if  the  responses  are  restricted  by  both  q  and  c  the  symmetrical  functional 
model  becomes 

Nf  =  kp-q  -  c  kp“^-1 
»  kp-q  <1  -  £) 

The  final  experimental  model  is  defined  as 

N  *  n  Nf 

for  the  symmetrical  case,  where  n  is  the  number  of  replications.  All  types 
of  matrix  experiments  can  be  described  by  such  models. 

The  unified  procedure  for  selecting  alternate  experimental  designs  can  be 
summarized  as 

1 .  Determine  experimental  design  criteria 

2.  Synthesize  the  experimental  model 

3.  Compare  model  to  standard  experimental  designs  and  choose  the 
optimal  design. 
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A  PROBLEM  IN  CONTINUOUS  SAMPLING  VERIFICATION 
Mary  E.  Bloae 

U , S .  Army  Ammunition  Procurement  and  Supply  Agency 
Joliet,  Illinois 

There  are  basically  two  types  of  sampling  Inspection  procedures  In  use 
today.  These  are  lot-by-lot  and  continuous  sampling  procedures.  In  addition 
to  the  two  types  of  sampling  Inspection,  there  are  also  two  different  methods 
of  Inspection,  namely,  by  attributes  and  by  variables.  Inspection  by  attributes 
is  on  a  go-no-go  basis.  That  is,  a  unit  of  product  Is  inspected  and  determined 
to  be  either  satisfactory  or  unsatisfactory  with  respect  to  the  characteristic 
under  consideration.  Under  Inspection  by  variables,  the  actual  value  of  the 
measurement  of  e  meeaureble  characteristic  is  rscorded.  Several  of  these 
measurements  might  then  be  uaad  together  to  aatiaate  some  parameter  upon  which 
a  lot  of  product  may  be  judged  relative  to  Ite  conformity  to  specification 
requirements.  Our  discussion  shall  be  limited  to  Inspection  by  attributes. 

Under  Inspection  by  attributes,  the  Inspection  can  be  performed  on  a  lot- 
by-lot  basis  or  continuously.  Let  us  flrat  consider  the  lot-by-lot  case.  The 
units  of  product  ars  divided  Into  Identifiable  lots,  and  a  lot  is  judged  either 
conforming  or  nonconforming  on  the  basis  of  the  number  of  defective  units 
found  In  a  sample  from  the  lot. 

One  of  the  most  widely  used  Military  Standards  listing  sampling  plans 
for  this  type  of  Inspection  Is  MIL-RTD-105D,  "Sampling  Procedures  and  Tables 
for  Inspection  by  Attributes,"  29  April  1963.  Hhan  using  this  Standard,  a 
sampling  plan  is  detsrminad  by  the  following t 
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*•  the  size  o-'  the  let, 

b.  the  specified  acceptable  quality  level  (AQL), 

c.  the  specified  Inspection  Level  (when  none  Ls  specified.  Inspection 
Level  II  ls  used),  and 

d.  the  type  of  plan  specified  or  approved  for  use  (single,  double, 
or  multiple) . 

The  size  of  the  lot  may  be  stated  in  the  specifications,  or  it  may  be  up  to  the 
supplier,  subject  to  approval  by  the  consumer,  to  determine  a  suitable  lot  size. 
The  AQL  is  the  maximum  percent  defective  of  product  which  can  be  considered 
satisfactory  for  the  process.  For  example,  in  this  Standard  possible  AQL  values 
are  .0102,  1.0X  and  10Z.  Once  a  plan  has  been  determined,  the  plan  parameters 
(sample  sizes  and  acceptance  and  rejection  numbers)  can  be  found. 

As  an  example  of  a  lot-by-lot  plan,  consider  a  single  sampling  plan  where 
the  lot  size  is  1000,  the  semple  size  is  100,  the  acceptance  number  is  3  and 
the  rejection  number  ls  4.  Then,  under  thin  plan  a  random  sample  of  100  units 
would  be  selected  from  the  lot.  The  number  of  defective  unitB  would  be  counted, 
and  if  the  number  were  3  or  less  the  lot  could  be  submitted  to  the  consumer 
for  acceptance.  If,  however,  the  number  of  defective  units  in  this  sample  vere 
4  or  more,  then  the  lot  could  not  be  submitted  to  the  consumer  for  acceptance, 
and  it  must  be  rejected. 

Let  us  now  turn  our  attention  to  continuous  sampling  inspection.  A 
limited  Standard  which  defines  various  types  of  these  sampling  plans  is 
MIL-STD-1235(ORD) ,  "Single  and  Multilevel  Continuous  Sampling  Procedures  and 
Tables  for  Inspection  by  Attributes,"  17  July  1962.  It  is  a  limited  Standard 
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In  that  it  is  applicable  only  to  the  Army.  This  Standard  is  a  composite 
of  Inspection  and  Quality  Control  Handbooks  (Interim)  H106,  "Multi-level 
Continuous  Sampling  Procedures  and  Tables  for  Inspection  by  Attributes," 

31  October  1958  and  H107,  "Single-level  Continuous  Sampling  Procedures  and 
Tables  for  Inspection  by  Attributes,"  30  April  1959. 

In  order  to  use  these  plans  the  following  criteria  must  be  met: 

a.  the  units  of  product  must  be  moving,  which  means  that  they  must 
pass  by  the  inspection  station  by  means  of  a  conveyor  belt  or 
some  other  conveyance,  such  as  a  tote  box  or  skid, 

b.  the  process  must  produce  homogeneous  material  or  be  capable  of 
producing  homogeneous  material, 

c.  there  must  be  relative  ease  of  inspection,  and 

d.  there  must  be  ample  physical  facilities  for  rapid  100Z  inspection. 

All  continuous  ssmpllng  plans  are  characterised  by  periods  of  screening 

and  sampling.  The  simplest  CSP  plan  is  designated  CSP-1  and  was  developed 
by  Dodge  (See  Annals  of  Mathematical  Statistics.  Sept.,  1943).  Under  this  plan, 
100Z  inspection  (screening)  is  performed  until  i  consecutive  good  units  have 
passed  Inspection.  The  prescribed  value  "i"  may  be  some  value  between  4  and 
2000,  depending  upon  the  specific  plan  being  used.  After  i  consecutive  good 
units  have  passed  inspection,  sampling  is  begun  at  a  certain  prescribed 
frequency,  f.  The  value  of  f  may  be  some  value  between  1/2  and  1/200,  again 
depending  upon  the  specific  plan  being  used.  Since  each  unit  of  product  should 
have  an  equal  chance  of  being  selected,  the  Interval  between  the  sampled  units 
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.should  vary  somewhat.  Sampling  Is  continued  until  a  defective  unit  is 
found.  When  this  occurs,  screening  (10QZ  inspection)  begins  and  continues 
until  1  consecutive  good  units  have  passed  inspection,  at  which  time  sampling 
will  again  be  introduced. 

A  sampling  plan  under  MIL-STD-1235(ORD)  is  also  determined  by  the 
fallowing  factors: 

a.  the  number  of  units  in  a  production  interval, 

b.  the  specified  A(J,,  and 

c.  the  specified  Inspection  Level  (when  none  Is  specified.  Inspection 
Level  II  is  used)  and 

d.  the  type  of  continuous  sampling  plan  specified  or  approved  for  use 
(CSP-1  or  one  of  the  other  types  of  plana  provided  In  the  Standard) . 

The  production  Interval  la  that  period  of  time,  usually  a  day  or  shift, 
during  which  conditions  of  manufacture  can  reasonably  be  expected  to  remain 
stable.  Of  the  four  continuous  sampling  procedures  provided  in  MIL-STD-1235(0RD) 
CSP-1  is  the  simplest.  It  will  be  the  only  one  considered  here. 

as  an  example  of  a  CSP-1  plan,  consider  one  In  which  i»20  and  f*l/10. 

Screening  would  be  performed  until  20  consecutive  good  units  had  passed  inspection. 
When  this  had  been  accomplished,  sampling  could  bagln  at  the  rate  1  in  10.  This 
means  that  the  sampling  inspector  would  select  1  out  of  10  units  but  would  vary 
the  interval  between  these  selected  units  to  give  each  unit  of  product  an 
equal  chance  of  being  included  in  the  sample.  Sampling  would  continue  until 
a  defective  unit  is  found.  At  that  time  scraening  would  again  be  instituted, 
and  it  would  be  necessary  to  screen  20  consecutive  good  unite  before  sampling 
could  be  resumed  again. 


Verification  of  the  supplier's  Inspection  records  is  advantageous  to 
the  consumer  because  he  would  like  to  ascertain  that  the  supplier  is  following 
the  Inspection  plan  and  classifying  inspected  units  properly.  That  is, 
inspected  units  which  are  defective  should  be  classified  defective  and  Inspected 
units  which  are  non-defective  should  be  classified  non-defective.  In  order 
to  achieve  this  aim,  AMSMU-P-715-503,  "Army  Ammunition  Plant  Quality  Assurance 
Procedures,"  December,  1966,  describes  the  appropriate  procedures  to  be  used 
by  Army  Ammunition  Plants  for  verification  purposes.  This  document  is  designed 
to  be  used  in  conjunction  with  either  lot-by-lot  or  continuous  sampling 
inspection,  and  can  therefore  be  used  with  MIL-STD-105D  or  MIL-STD-1235(ORD) . 

In  the  lot-by-lot  case,  it  is  a  relatively  easy  matter  to  perform  verification. 
First,  the  supplier  selects  a  random  sample  from  the  lot  in  question  and  counts 
the  number  of  defective  units  in  this  sample.  He  then  compares  the  number  of 
defective  units  to  the  acceptance  number  for  his  specified  sampling  plan  from 
MIL-STD-105D .  If  the  number  of  defectives  is  equal  to  or  leas  than  the 
acceptance  number,  the  lot  may  be  submitted  to  the  consumer  for  acceptance. 

The  consumer  takes  a  sample  from  the  lot,  and  counts  the  number  of  defective 
units.  The  consumer  is  then  ready  to  compare  his  results  with  those  of  the 
contractor  using  Table  I  of  Quality  Control  and  Reliability  Handbook  (Interim) 
H109,  "Statistical  Procedures  for  Determining  Validity  of  Suppliers'  Attributes 
Inspection,  "  6  May  1960.  For  purposes  of  this  comparison,  it  is  assumed  that 
the  consumer  has  classified  all  of  his  sample  units  properly.  The  H-109 
comparison  is  in  effect  a  test  of  significance  between  the  number  of  defectives 
found  by  the  supplier  and  the  number  of  defectives  found  by  the  consumer,  given 
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a  certain  value  r,  which  is  the  ratio  of  the  supplier's  sample  size  to  r,-,e 
consumer's  sample  size.  Rejection  under  this  test  will  cause  the  supplier 'j 
data  to  be  considered  Invalid. 

Verification  of  Inspection  results  when  the  sampling  inspection  Is  done 
by  continuous  sampling  procedures  is  more  complicated.  Under  the  provisions 
of  MIL-STD-1235(0RD)  and  AMSMU-P-715-503,  the  supplier  performs  checking 
inspection  at  rate  f  luring  all  periods  of  screening,  in  order  to  ascertain, 
that  the  screening  ere*  is  doing  an  efficient  job.  The  units  inspected  during 
this  checking  Inspection  plus  the  units  inspected  by  the  supplier's  sampling 
inspector  form  the  supplier's  ssmple  for  comparison  purposes,  where  the  period 
under  consideration  is  a  production  interval. 

Concurrently  with  the  inspection  by  the  supplier  described  above,  the 

J'  i 

consumer  is  performing  verification  inspection  at  rata  (l/r)f,  where  r  is  the 
ratio  of  comparison  sample  sizas  described  previously  and  f  is  the  prescribed 
sampling  frequency.  The  method  of  determining  the  particular  value  of  r  (1,  2, 
3,  5  or  8)  to  be  used  le  outlined  in  AMSMU-P-715-503  end  is  not  important  to 
our  discussion  hers,  since  we  will  only  concern  ourselves  with  the  cess  r-8. 

The  various  types  of  inspection  described  above  ere  summarized  in  Table  1. 
Reviewing  the  Table,  and  from  the  preceding  dlucusslon,  it  can  be  noted  that 
only  one  type  of  inspection  is  performed  by  the  consumer,  namely,  verification 
inspection,  and  this  la  done  at  a  definite  sampling  frequency  which  le 
proportional  to  that  used  by  the  supplier  (in  the  case  to  be  considered  here, 
the  proportion  is  one-eighth) .  The  unite  inspected  in  this  manner  constitute 
the  consumer's  verification  sample  which  is  used  for  comparison  purposes  with 
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■  . '....lied  by  the  supplier .  Again,  th“  snmp1<*.  «M.~n  frvr  ennvenfenct  s.u  <.  1 

be  _alleJ  the  comparison  sample  hereafter,  is  comprsed  ot  anits  vh*ch 
may  have  come  from  the  screening  or  sampling  phase  with  the  proportion  of 
units  from  any  phase  for  a  production  interval  dependent  upon  the  amount  of 
time  spent  on  this  phase  by  the  supplier.  The  consume!  usually  has  no  knowl¬ 
edge  as  to  which  units  came  from  which  phase  since  verification  inspection 
might  be  performed  at  a  place  far  removed  from  the  inspection  conducted  by 
the  supplier. 

Let  ut>  consider  how  these  inspections  function.  Since  we  are  considering 

on  y  continuous  operations  under  CSP-1  procedures,  the  units  of  product  will 
be  moving  past  the  various  inspection  stations  via  conveyor  belts,  tote  boxes 
or  some  other  conveyance.  Let  us  flrat  consider  the  supplier's  function.  As 
the  operation  begins,  the  product  Is  Inspected  100%  to  remove  any  defective  units 
and  to  nee  if  i  consecutive  good  units  can  be  found.  Concurrently  with  this 
Initial  product  inspection  la  checking  Inspection  which  In  performed  at  a  rate 
f  (the  specified  sampling  frequency)  and  is  a  means  of  checking  the  effectiveness 
of  the  screening  operation.  The  units  sampled  during  this  checking  inspection  will 
form  part  of  the  supplier's  comparison  sample.  Once  i  consecutive  good  units 
have  been  found,  sampling  Inspection  of  the  product  is  initiated.  This  sampling 
of  the  units  of  product  is  done  in  a  random  manner  at  some  specified  sampling 
frequency,  f.  The  units  sampled  form  the  remainder  of  the  supplier's  comparison 
sample. 

Let  ub  now  review  the  consumer's  inspection  function.  As  can  be  seen  from 
Table  I,  there  is  only  one  type  of  inspection  which  the  consumer  performs, 
namely,  verification  inspection.  This  inspection  is  done  concurrently  with 
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tne  supplier's  inspection.  The  point  at  which  the  consumer  conducts  this 

Inspection  nay  be  far  ramoved  from  the  site  of  the  supplier's  inspection  opera- 

» 

tions.  Since  the  units  of  product  are  not  marked  or  designated  as  to  which 
units  came  from  which  phase,  the  consumer  generally  is  Ignorant  of  this 
information.  The  coneumer  samples  Che  unite  In  a  random  manner  at  a  sampling 
frequency  which  ia  proportional  to  tha  sampling  frequency  used  by  the 
supplier.  This  value  of  the  sampling  frequency  is  [(l/r)(f)],  where  1/r 
is  the  proportional  factor  (one-eighth  for  purposes  of  discussion  here)  and  f 
is  the  prescribed  sampling  frequency.  Because  the  sampling  is  done  in  a 
random  manner  without  requiring  a  certain  number  or  percentage  of  the  in¬ 
spected  unite  to  be  from  any  one  phase,  there  might  be  a  considerable 
difference  in  the  proportion  of  unite  from  one  of  the  phases  for  the 
consumer  end  supplier  during  the  production  Interval. 

To  use  Table  I  of  H-109  to  compare  d,(  ■  d^gg  +  da,f)  vich  dc 
(  -  dc , 100  +  dC(f/g),  the  probability  of  accepting  the  hypothesis  of 
validity  should  remain  the  same  as  reflected  on  the  O.C.  curves  (3ee  Figure 
I,  extracted  from  H-109)  for  the  test  to  be  of  the  level  a  end  probability 
of  acceptance  over  the  parameter  space  ae  shown  on  ths  O.C.  curves.  By 
way  of  sxplanation  tha  paramater  under  consideration  is  the  ratio  of  fractions 
defective,  pc/pg,  which  can  be  thought  of  as 

Prob  (defective  lnepec.ted  unit  will  be  claselflad  defective  by  consumer) 

Prob  (defective  inspected  unit  will  be  classified  defective  by  supplier) 

This,  then,  la  our  problem*  To  show  that  tha  probability  of  accepting 
the  hypothesis  of  validity  over  the  parameter  space  is  approximately  the 
same  as  that  shown  on  the  O.C.  curves. 
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To  simplify  the  remainder  of  the  discussion  and  the  problem 
definition,  che  notation  below  shall  be  used. 

Let 

ns,100  “  number  of  units  in  supplier's  comparison  sample 
coming  from  the  screening  phase, 

ng>f  ■  number  of  units  In  supplier's  comparison  sample 
coming  from  the  sampling  phase, 

^  100  m  rumher  of  units  in  consumer's  comparison  sample 
ciming  from  the  screening  phase, 

n£  fyg  -  number  of  units  in  Consumer's  comparison  sample 
*  coming  from, the  sampling  phase. 

Let  d,  subscripted  as  above,  refer  to  the  number  of  defective  units  found 

in  the  portion  of  units  identified  by  the  subscripts. 

Let  ua  now  reflect  on  some  aspects  of  the  problem. 

Since  there  are  two  phases,  namely,  the  screening  phase  and  the  sampling 
phase,  from  which  the  verification  sample  as  well  as  the  supplier's  comparison 
sample  can  come,  there  is  a  possibility  of  considerable  variation  between 
the  two  in  the  proportion  of  units  from  any  one  phase.  That  is,  for  example, 

n8,100 

— '"T  ■_ —  might  be  considerably 

ns , 100  +  ns,f 


different  from  ^lOO 

nc,100  +  "e,f /8 

Let  us  now  consider  only  one  value  of  the  parameter  space,  pc/pa  ■  1. 
which  is  equivalent  to  saying  that  the  supplier  has  perfect  inspection 
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efficiency.  Then  no  defectives  should  be  found  in  the  samples  from  the 
screening  phase  since  these  should  have  been  removed  during  the  screening 
phase  of  product  Inspection.  Hence,  any  defective  which  would  be  found 
in  either  of  these  samples  would  come  from  the  sampling  phase. 

4?  reflection  will  show,  if  the  units  comprising  the  samples  were 
selected  completely  independently  of  order  or  position  in  the  production 
interval,  we  would  have  a  situation  equivalent  to  a  lot -comparison  situation., 
and  the  O.C.  curves  would  be  exactly  as  defined  for  H-109.  Further,  if 
the  proportions  described  previously  were  exactly  the  same,  that  is,  the 
fraction  of  the  supplier's  comparison  sample  coming  from  the  screening 
phase  were  exactly  the  same  as  the  fraction  of  the  consumer's  phase,  we 
would  have  essentially  a  stratified  sampling  problem,  and  again  the  O.C, 
curves  would  be  exactly  as  defined  In  Hrl09. 

Since  the  prescribed  method  of  sampling,  however,  is  to  take  about  one 
out  of  every  1/f  units,  allowing  the  interval  between  inspected  units  to 
vaty  somewhat,  we  have  neither  of  the  situations  described  above.  This 
brings  us  to  the  reason  why  we  are  only  considering  the  case  r«8.  It 
is  reasonable  to  assume  that  the  greatest  variation  from  the  O.C.  curves  of 
H-109  is  possible  for  the  largest  value  of  r.  Therefore,  if  this  variation 
is  insignificant  for  r-8  it  should  be  insignificant  for  the  lower  value 
of  r.  Let  us  now  consider  a  specific  example. 

Since  screening  need  only  be  done  at  the  initiation  of  production,  and 
thereafter  only  when  a  defect  is  found  during  a  period  of  sampling  inspection, 
it  is  not  necessary  in  our  example  to  assume  that  screening  is  initiated  at 
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'i'w  suit  of  the  production  interval,  but  for  sake  of  discussion  let  us 
assume  that  it  aces.  Suppose  the  supplier  is  sampling  h.  a  frequency  of 
1/10,  and  the  consumer  is  using  a  ratio  of  r*8.  Therefore,  the  consumer 
would  be  sampling  at  a  frequency  of  1/8Q.  First,  the  supplier's  screening 
crew  Inspects  all  units  of  product  until  the  appropriate  number  of  consecutive 
good  units  has  been  cleared.  At  the  same  time,  the  checking  inspector 
is  selecting  one  unit  out  of  ten  in  a  random  manner  to  see  if  the  screening 
r.rtv  is  doing  its  job  p-o-frly.  After  the  necessary  n-inbet  of  consecutive 
good  units  has  been  cleared,  sampling  inspection  is  begun  whereby  one  out 
of  ten  units  is  selected  for  inspection.  There  is  no  checking  inspection 
during  this  phase. 

During  the  entire  production  interval,  the  consumer’s  verification  inspector 
selects  one  out  of  eighty  units  ih  a  random  manner.  At  the  completion  of  the 
production  interval,  the  supplier's  and  consumer's  comparison  sample 
inspection  results  can  be  compared.  The  supplier's  sample  consists  of  those 
units  inspected  by  the  checking  Inspector  during  the  screening  phase  plus 
the  units  inspected  by  the  supplier  during  the  sampling  phase.  The 
consumer's  sample  consists  of  all  units  inspected  by  the  verification 
inspector,  whether  these  units  came  from  the  screening  or  sampling  phase. 

Let  us  assume  that  the  production  interval  encompasses  80  units  and  76 
of  these  units  were  subjected  to  screening  while  the  remaining  4  units  were 
part  of  the  sampling  phase.  Let  ub  suppose  the  sampling  frequencies  are  as 
above,  namely,  f “1/10  for  the  supplier  and  f-1/80  for  the  consumer. 
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Rer xectlon  will  show  that  there  are  many  possible  variations  In  the 
values  of  n8tl00,  ng>f,  n  qq,  ***  nc,f/8‘  Ifc  is  P08slble»  for  example, 

-liat  all  of  the  units  for  the  supplier's  sample  came  from  the  screening 
phase  while  the  single  unit  composing  the  consumer's  sample  came  from  the 
sampling  phase.  In  this  case,  the  proportion  of  units  in  the  supplier's 
sample  from  the  screening  p\  isa  is  1.0  whereas  the  corresponding  proportion 
of  units  in  the  consumer's  sample  from  the  same  phase  is  0. 

Since  the  probability  oj  each  possible  variation  is  not  known,  since 
strict  probabilistic  sampling  la  not  performed,  the  effective  O.C.  curve 
cannot  he  determined  simply. 

Ideally  then,  a  mathematical  model  describing  the  O.C.  curves  would 
be  desirable. 

In  lieu  of  such  a  mathematical  model,  ve  conducted  a  Monte  Carlo  simulation 
of  the  process.  Twenty  different  simulations  of  various  CSP-1  and  CSP-2 

plans  were  considered.  A  few  selected  AQL's  ranging  from  0.01%  to  4.0%  were 

S 

used,  with  production  Intervals  ranging  from  70  units  to  1000  units.  The 
value  of  p  was  set  equal  to  the  AQL  in  each  case  on  these  first  attempts. 

Ten  production  intervals  were  considered  for  each  simulation.  Finally,  it 
was  assumed  that  the  screening  crew  was  100%  efficient,  i.e.,  all  defective 
units  were  removed  during  the  screening  phase. 

Random  numbers  were  used  to  designate  the  defective  unite.  Once  this 
had  been  determined,  the  Inspection  processes  could  be  simulated.  First, 
the  units  from  the  initial  screening  phase  were  identified,  and  then  random 
numbers  were  used  to  select  the  first  unit  to  be  sampled  by  the  supplier. 
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for  convenience  on  these  first  attempts,  a  systematic  sample  followed  the 
selection  of  the  first  unit.  When  all  ten  production  intervals 
n ad  been  completed  in  this  manner,  the  units  inspected  by  the  checking 
inspector  during  the  screening  phase  needed  to  be  specified.  Rendon 
numbers  were  again  employed  to  designate  the  initial  units  sampled  during 
these  phases  and  systematic  sampling  ensued.  When  this  had  been  completed, 
tne  proportion  of  unlt3  from  the  screening  phase  for  each  production 
interval  and  for  the  tei  production  intervals  as  a  whole  could  be  calculated, 
lien,  the  consumer's  Inspection  had  to  ba  simulated.  Since  the  sampling 
was  done  at  a  specified  sampling  frequency  without  regard  as  to  which  phase  the 
supplier  was  on,  a  random  number  was  used  to  indicate  the  first  unit  of 
the  sample,  and  a  systematic  sampling  followed  for  the  duration  of  the  ten 
production  intervals.  Upon  the  completion  of  the  ten  production  Intervals,  the 
proportion  of  units  from  the  screening  phase  for  each  production  interval  and 
far  the  ten  production  intervals  as  a  whole  could  be  tallied.  Thaaa 
proportions  could  then  be  compared  to  the  corresponding  one  for  the  supplier. 
Table  2  shows  the  results  of  one  of  these  simulations.  For  this  simulation, 
the  production  size  was  70;  the  AQL  was  2.51;  the  i  value  was  25;  the 
supplier's  sampling  frequency  was  1/5,  and  the  consumer's  sampling  frequency 
was  1/40.  There  does  not  appear  to  be  too  much  difference  between  the 
proportions  except  for  the  seventh  production  Interval  where  the  supplier's 
proportion  was  .357,  and  the  consumer's  proportion  was  0. 

In  order  to  use  the  O.C.  curves  from  H-1Q9 ,  acme  calculations  needed  to 
be  performed.  The  fractions  defective  for  the  supplier  and  consumer  as 
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well  as  Che  expected  number  of  defective  units  in  the  supplier's  sample 
needed  to  be  specified.  Since  it  was  assumed  that  the  screening  crew 
was  JOOZ  efficient,  theoretically  no  defective  units  should  have 
appeared  in  either  the  supplier's  or  the  consumer's  sample  from  the 
screening  phase.  Therefore,  the  fraction  defective  for  either  the  supplier 
or  consumer  is  the  proportion  of  units  from  the  screening  phase  times 
the  appropriate  AQL  (since  p  was  set  equal  to  the  AQL,  as  mentioned 
previously).  Then,  the  ratio  of  the  consumer's  fraction  defective  to 
the  supplier’s  fraction  defective  was  calculated.  Finally,  the  expected 
number  of  defective  units  in  the  supplier's  sample  was  estimated  by  the 
number  of  units  in  the  production  Interval  times  the  fraction  defective 
described  above.  The  results  of  these  computations  for  each  of  the 
ten  production  intervals  and  for  the  ten  production  intervals  as  a  whole 
are  summarised  In  Table  3.  The  last  two  columns  are  of  more  interest. 

It  will  be  noted  thst  most  of  the  ratios  are  around  1.0  axcept  for  pro¬ 
duction  Interval  #7  whare  the  ratio  is  2.6040. 

Note  that  all  of  the  expected  number  of  defective  unite  in  the  supplier's 
•ample  for  our  example  are  considerably  lass  than  tha  amalleat  value, 
indexing  the  H-109  curves  (see  figure  at  end  of  paper),  namely,  0.75. 

Hence,  the  O.C.  curves  for  these  figures  would  be  above  that  for  0.75. 

Also,  some  of  our  ratios  are  leaa  than  1.0  which  is  the  smallest  ratio 
given  on  the  chart.  This  means  that  the  probability  of  acceptance  for 
these  ratios  would  be  even  greater  than  0.95  which  la  the  corresponding 
value  when  the  ratio  is  1.0. 
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iJhile  we  were  unable  to  develop  a  suitable  model  to  determine 
whether  the  probability  of  acceptance  over  the  long  run  would  be  of 
any  Important  difference  from  that  yielded  by  the  h-109  O.C.  curve 
formula  described  by  Ellner  (see  Technometrics.  February  1963,  pp.  23-46) 
it  seemed  reasonable  to  assume  that  if  the  variation  of  individual 
simulation  results  from  the  H-109  value  were  small,  the  probability  of 
acceptance  under  tie  continuous  sampling  verification  method  could  be 
adequately  describe.'  Lv  the  Ellner  formula. 

To  simplify  our  work,  we  arbitrarily  decided  to  concern  ourselves 
only  with  the  frequency  of  simulation  for  which  the  probability  of 
acceptance  was  less  than  .90.  .  This  would  allow  us  to  get  a  quick  picture 
of  the  results  without  having  to  compute  an  O.C,.  curve  point  for  each 
simulation. 

If  we  consider  all  of  the  production  Intervals,  it  can  easily  be 
seen  that  they  meet  the  criterion  of  having  a  probability  of  acceptance 
of  greater  than  .90.  Therefore,  in  this  example,  it  seems  reasonable 
to  assume  that  the  O.C.  curve  under  the  continuous  sampling  assumption 
is  probably  close  to  the  range  of  values  (94Z-96X)  provided  by  the  Ellner 
formula. 

Thus,  it  is  possible  to  study  this  problem  using  simulation  methods. 
However,  it  obviously  would  be  preferable  to  have  a  mathematical  model. 
Therefore,  to  reiterate  the  problem:  a  mathematical  model  describing 
the  operating  characteristic  of  the  procedure  described  is  desired. 
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TABLE  II 


PROPORTION  OF  UNITS  SUBJECTED  TO  100/.  INSPECTION 


Production 

Screening  Phase 

Interval 

Supplier 

Consumer 

1 

0.357 

0.500 

2 

0.000 

0.000 

3 

0.Q00 

0.000 

4 

0.000 

0.000 

5 

0.000 

0.000 

6 

0.000 

0.000 

7 

0.357 

0.000 

8 

0.357 

0.50Q 

9 

0.067 

0.000 

10 

0.308 

0.500 

Cumulative 

0.143 

0.150 
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TABLE  HI 


Production  Interval 

Pef£c 

Peffs 

Pc 

P. 

“«P. 

(expected  nmber  of 
defectives  in  supplier's 
sample) 

1 

.0125 

.0161 

.7760 

.2250 

2 

.0250 

.0250 

1.0000 

.3500 

3 

.0250 

.0250 

1.0000 

.3500 

4 

.0250 

.0250 

1.0000 

.3500 

5 

.0250 

.0250 

1.0000 

.3500 

6 

.0250 

.0250 

1.0000 

.3500 

7 

.0250 

.0096 

2.6040 

.1340 

8 

.0125 

.0096 

1.3020 

.1340 

9 

.0250 

.0230 

1.0900 

.3450 

10 

.0125 

4 

.0173 

.7225 

.2250 

Cumulative 

.021 

.021 

1.00 

.294 
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PROBABILITY  OF  ACCEPTANCE 


OPERATING  CHARACTERISTIC  CURVES 
OF  TWO-SAMPLE  TEST  FOR  HOMOGENEITY 


SAMPLE  SIZE  RATIO;  r  =  b 


»ATlO  OF  FRACTIONS  DSFCCTIVC, 


NOTCi 

Flgurci  on  curve*  ere  the  expected  number*  el  Selective!  (defect!)  In  tKe  *uppli*r'*  tempi 
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ABSTRACT 

We  present  an  account  of  an  attempt  to  find  useful  random  models  of 
terrain.  Measurements  have  shown  that  tha  distribution  of  slopes  is  what 
has  bean  called  the  bilateral  exponential  distribution,  definitely  not 
normal.  Tha  problem  ia  to  find  a  convenient  random  function  of  geographical 
positions  of  two  real  variables  which  has  this  distribution  for  slopes  and 
fits,  in  some  approximations,  tha  dependence  of  slopes  in  various  directions 
at  neighboring  points.  A  family  of  random  functions,  the  probability 
distributions'  in  function  space  which  are  spherically  symmetric  in  a 
Hilbert  norm  suitable  to  the  purposes  of  tha  study,  was  introduced  with 
on  enormous  latitude  in  the  choice  of  parametric  functionals.  We  felt 
sura  that  random  functions  with  tha  required  properties  must  be  included. 

Sad  to  relate  further  mathematical  developments  which  wa  deem  intrinsically 
interesting  have  shown  it  not -to  be  so.  We  know  not  hew  to  proceed.  Help! 


This  article  has  been  reproduced  photographically  from  the  authors’  manuscript. 

Preceding  page  blank 
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We  have  found  it  easier  to  separate  our  contributions  to  this 
exposition  although  they  are  interdependent.  Peterson  has  written  the 
first  paragraphs  and  Taylor  the  later  ones  as  is  indicated  in  the  text. 

Terrain,  being  the  medium  of  ground  combat,  has  been  the  subject 
of  many  investigations  by  analysts  in  the  field  of  weapon  systems  analysis. 
Most  of  these  studies  have  been  focused  on  the  particular  role  played  by 
terrain  in  the  particular  problam  at  hand.  Othars  have  been  more  general 
in  nature  with  a  goal  of  giving  more  insight  into  the  quantitative  aspects 
of  describing  terrain. 

I  would  like  o  indicate  a  sample  of  the  t  pe  of  problems  that 
arise  involving  terrain  and  its  influence  on  the  ou  come  of  combat  that 
have  received  attention.  In  order  to  lend  some  at  "it lance  of  order  to 
such  a  listing  I  have  attempted  this  simple  two  way  classification  of  thaae 
roles.  (Figure  1)  For  lack  of  better  terma  I  have  labeled  them  scales 
and  mechanisms .  For  scales  I  have  fallen  back  on  tk  e  vernacular  of  micro 
and  macro,  micro  generally,  referring  to  distances  of  up  to  a  few  meters  and 
macro  from  there  on  out  to  perhaps  several  kilometers.  Mechanisms  I  have 
broken  into  a  clear  dichotomy  of  contact  and  non-contact.  By  contact  I 
mean  that  the  terrain  is  actually  supporting  the  objects  whether  they  be 
vehicles  or  other  piecea  of  equipment  being  considered.  By  non-contact  I 
mean  we  are  concerned  with  the  existence  of  a  line-  if-aight.  I  have  listed 
those  roles  of  terrain  which  can  ba  fairly  wall  categorized  but  I  will  also 

try  to  indicate  problam  areas  where  there  ie  not  a  clear  distinction  or 

.1 

there  afe  strong  interactions . 

Under  contact  with  the  micro  structure  of  terrain  I  have  listed 
VRIDE  an<*  Postur*a  vriqe  has  come  to  be  usod  as  an  indication  of  the 
speed  of  a  vehicle  that  is  tolerable  to  both  the  occupants  and  to  the 
vehicle  itself  due  to  roughness  of  tarrain.  It  is  concerned  with  dynamics 
of  the  vehicle  over  the  tarrain. 
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FIGURE  1 


* 

ROLES  OF  TTRRAIN  IN  COMBAT 
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Posture  refers  more  to  the  static  role  of  the  vehicle  and  is 
conaemti  with  the  capability  of  a  weapon  system.  As  an  nvamnl*  If 

-  -  *  r 

a  tank  is  canted  and  the  gun  is  elevated,  this  elevation  will  introduce 
a  horizontal  component  of  error  into  the  aim  of  the  gun. 

Under  contact  with  the  macro  characterietics  of  terrain  I 
have  listed  simply  routes.  Factors  other  than  slopes  influence  the 
routes  taken  by  a  vehicle,  of  course. 

Under  non-contact  in  the  micro  regime  I  have  indicated  the 
fragment  shielding  which  has  been  quantified  in  tenss  of  "cover  functions". 

The  non-contact  aspects  of  the  macro  relief  are  closely  tied  up  with  the 
ranges  of  engagement.  A  defender  may  choose  fields  of  fire  to  get  his  opponent 
out  into  the  open  and  yet  there  may  be  draws  and  gullies  which  can  allow 
the  attacker  to  approach  under  cover. 

As  an  example  of  the  multiple  interaction  of  all  of  these  roles, 
we  might  consider  the  case  of  a  tank  hastily  taking  up  a  firing  position. 

The  tank  is  advancing  along  boom  preplanned  axle  --  his  route  has  been 
established.  The  enemy  is  encountered  —  the  approximate  range  of  engagement 
has  been  established.  The  tank  may  stop  or  head  for  a  nearby  rise  in  the 
terrain  to  get  into  hull  defilade  —  VRIDE  and  shielding  come  into  play. 

How  the  tank  is  canted  in  position  may  influence  hia  accuracy  —  hence  the 
role  of  posture. 

He  see  than  that  there  are  a  number  of  properties  of  terrain  that 
are  of  concern  to  the  military  OR  analyst  and,  as  I  have  mentioned  at  the 
outset,  there  are  a  number  of  ways  that  terrain  has  been  categorized, 
measured,  stored  in  machine  memories  for  retrieval,  generated  by  Monte 
Carlo  means,  etc.  In  order  to  stats  th#  problem  which  we  bring  to  this 
clinical  session  I'd  like  to  discuss  two  observations  concerning  the  nature 
of  terrain  which  we  feel  have  not.  been  exploited  to  their  fullest  in  dealing 
with  this  problem  area.  One  concerns  the  restate  of  e  statistical  study  of 
terrain  slope.  The  other  concents  the  underlying  geometry  of  the  nature  of 
terrain. 
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The  statistical  study  to  which  I  refer  was  conducted  to  determine 
the  posture  of  tanks  as  measured  by  the  pitch  and  cant  of  the  trunnion  after 
taking  up  simulated  firing  positions.  A  sampling  of  widely  varying  terrain 

P 

types  was  obtained  in  that  the  test  was  run  at  fort  Knox,  Fort  Bragg,  Fort 
Hood,  Camp  Pickett  and  Camp  Erwin.  The  pattern  that  emerged  indicated  that 
the  distributions  of  slopes  in  these  firing  positions  were  not  normal  but 
seemed  to  be  much  more  like  the  bilateral  exponential  distribution. 

(Figure  2)  Moreover  the  mean  absolute  slope  varied  greatly  from  one  test 
site  to  another.  In  order  to  check  out  the  possibility  that  this  non-normal 
characteristic  of  these  distributions  was  due  to  the  selection  of  the  firing 
positions  sample  profiles  of  each  of  the  test  sites  were  constructed  from 
maps  of  each  of  the  installations  and  the  distributions  of  slopes  measured 
over  200  yard  intervals  were  obtained.  Here  again,  the  bilateral  exponential 
distribution  seemed  to  be  the  natural  means  of  describing  these  slopes. 

A  detailed  map  study  of  the  type  mentioned  above  was  made  of 
the  region  around  Houffalize,  Belgium  (based  on  a  map  we  happened  to  have 
available).  It  showed  that  the  distributions  of  north-south  slopes  and 
of  east-west  slopes  both  seemed  to  fit  the  bilateral  exponential.  The 
inadequacy  of  the  normal  distribution  for  generating  profiles  from  which 
lines-of-sights  can  be  determined  was  demonstrated  some  20  years  ago  by 
people  in  the  U.  K.  (personal  communication  from  Mr.  Eddie  Benn  then  at 
the , Armament  Research  and  Development  Establishment).  This  finding  has 
seemed  to  influence  their  subsequent  investigations  along  this  line.  (See 
Forbes,  "The  Generation  of  Terrain  on  an  Electronic  Computer,"  A.R.D.E. 
Memorandum  (B)  75/60). 

In  several  of  the  studies  mentioned  above,  attempts  were  made 
at  establishing  distributions  of  the  height  or  elevation  of  terrain  itself. 
The  results  wore  erratic  and  no  pattern  was  observed.  Such  behavior  is 
probably  due  to  general  trends  which  can  be  attributed  to  near-zero 
frequency  components  in  the  spectrum. 


427 


FIGURE  2 


THE  FREQUENCY  FUNCTION  OF  THE 
BILATERAL  EXPONENTIAL  DISTRIBUTION 

£(«)  - 


where 

a  -  mean  absolute  deviation 

a2  -  o2/2 

characteristic  function 

4(a)  ■  [1  +  o2a2/2] 


428 


In  addition  to  the  non-normal  nature  of  terrain  there  is  the 

nTVlhl  ^»m  rtf  Himonc  l  rtn a  1  •?  fir  M anif  r\t  -t-hft  Av t  cfi'  nrr  e^Kfimao  frtn  rranAviaf  Inn 
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randan  terrain  profiles  proceed  in  the  same  manner  that  one  would  treat  a 
time  series.  This  approach  cannot  be  used  to  generate  a  surface,  as  two 
neighboring  rays  say  emanating  from  a  point,  will  be  completely  independent. 

Put  in  terms  of  statistically  describing  terrain  rather  than  generating  it 
we  must  think  in  terms  of  the  gradient  of  a  surface  rather  than  the  slope 
of  a  curve.  We  know  from  vector  analysis  that  the  curl  of  a  gradient  is 
zero.  In  other  words  there  are  constraints  between  the  two  perpendicular 
components  of  the  gradient  at  a  point.  In  the  one  dimensional  case,  as 
typified  by  a  time  seiies,  the  random  function  or  stochastic  process  is  readily 
expressed  in  terms  of  courier  series,  i.e.,  sines  and  cosines.  In  the  two 
dimensional  case  the  functions  which  replace  the  trigonometric  functions  in  a 
natural  way  are  the  Bessel  functions.  Other  areas  of  endeavor  on  which 
reference  to  two  dimensional  random  functions  have  been  found  include  windblown 
waves,  agricultural  productivity  and  images  both  photographic  and  video.  The 
household  term  of  enow  as  applied  to  a  television  picture  is  just  an  adoption 
of  tha  television  engineer's  term  "white  snow"  which  is  hia  extension  to  two 
dimensions  of  ths  concept  of  "white  noise"  in  the  one  dimensional  process. 

(We  might  add  in  passing  that  tha  most  wall  known  application  of  three 
dimensional  random  functions  ie  in  tha  field  of  turbulence . ) 

We  have  briefly  stated  two  characteristics  of  terrain  which  we 
believe  to  be  pertinent  to  the  statistical  description  of  terrain.  One 
baaed  on  data  analysis  that,  whereas  terrain  height  itself  doaa  not  stem 
to  have  any  pattern  to  its  distribution,  its  difference  field  as  mtasured 
over  a  few  maters  or  a  few  hundred  meters  has  a  comnon  non-normal  distribution 
which  can  be  expressed  in  terms  of  a  single  parameter.  Tha  other  baaed  on 
geometrical  reasoning  indicates  that  the  tools  developed  for  ono  dimensional 
processes  art  not  adequate  for  describing  a  two  dimensional  random  surface. 

We  are  now  at  a  point  of  being  able  to  state  the  problem  which 
baa  plagued  us  for  a  nunber  of  years.  Is  it  possible  to  construct  a 
meaningful  stochastic  model  of  terrain  which  eabodles  these  two  considerations? 
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•3u«-3tioTis  for  which  we  would  like  to  get  more  insight,  include  the 

1}  i c t  vc  categorize  cc  rough,  rc^lziig  or  flaw  in  .aCw 

simply  manifest*. i uns  of  the  same  basic  model  with  different  scale  factors 
in  the  horizontal  and  vertical  directions,  2)  to  what  extent  can  we  use 
easily  obtained  information  for  a  region  and  infer  the  details  from  the 
model  and/or  3)  can  we  build  a  composite  model  from  which  we  can  infer 

both  the  micro  and  macro  characteristics  of  a  given  terrain  type? 

/ 

In  closing  my  part  of  this  presentation  I  want  to  stress  that 
we  are  not  posing  the  general  question  as  to  how  to  statistically 
categorize  terrain  but  as  to  what  extent  the  theory  of  two  dimensional 
random  functions  can  cot  tribute  to  our  basic  understanding  of  |the  statistical 
properties  of  terrain.  , 

Dr.  Taylor  will  now  describe  one  approach  we  have  taken  to  this 
problem  along  with  its  triumphs  and  pitfalls.  I 

A  Claaa  of  Random  Functions 

After  careful  consideration  of  soma  requirements  on  a  random 
function  that  it  be  eligible  for  consideration  as  a  random  terrain,  Peterson 
was  led  to  propose  the  following  wide  class  of  random  functions  aa  candidates 
for  investigation.  Let  t, 

Ps  x  »  (x^.Xj)  (1.1) 

be  rectangular  coordinates  of  a  point  P  in  a  horizontal  datum  plana.  Lat 
u(x)  be  the  height  of  a  terrain  above  the  datum  plane  at  the  geographical 
point  P.  For  our  purpose  u(x)  is  a  complete  description  of  the  terrain. 

We  are  concerned  with  a  random  function  U(x),  a  probability  distribution 
on  certain  subeets  of  a  set,  aay  B,of  functions  u(x).  We  consider  A  linear 
set  B"  of  linear  functionals  Ku(O)  and  suppose  that  tha  expaction 

£(4(U)>  «  0  (1.2) 

for  all  l  of  the  eat.  By  the  variance  of  t  we  mean 

E(f(U)2)  ■  Var  L  (1.3) 
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and  Ly  Che  characteristic  functional  of  i  we  mean  the  expectation  oi  the 
exponential 

E(exp  U(U))  =  Ch  l  (1.4) 

The  proposal  is  to  limit  our  discussion  to  those  random  functions  for  which 
there  exists  a  complex  valued  function  of  a  real  positive  argument  g(z) 
such  that,  for  all  l , 

Ch*  =  g(Var  l)  (1.5) 

Example:  For  a  gauss lan  random  function  U, 

E( sxp  i  U))  =  exp( -  j  E(|f(U)|2)), 

sine*  E(l(U) )  =  0. 

Spherical  Symmetry 

We  may  introduce  also  tha  inner  product 

<tvl2>  *  (2.1) 

and 

l  2  «  <l%t> 

•  E(i(U)2) 

«  Var  l.  (2.2) 

It  Is  but  •  small  step  to  sxtand  our  discussion  to  the  Hilbert  space,  H, 
of  linear  functionals  and  to  suppose  further  that  this  space  is  sufficient 
in  the  following  sense:  For  any  u(x)  under  discussion 

£(u(x))  «  0  for  all  l  c  H  (2.3) 

implies  u(x)  s  o.  This  is  not  necessary  for  the  rather  loose  discussion 
we  are  presenting  but  it  may  ease  the  reader's  way.  Now  a  function  u(x) 
defines  on  H  a  linear  functional  whoaa  value  at  the  element  l  is  £(u). 
Whether  every  linear  functional  in  B*  is  thus  representsd  by  soma  function 
u(x)  is  of  no  importance  to  our  discussion •  What  is  vary  important  is  to 
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realize  that  the  linear  functional  defined  by  u(x)  need  not  be  in  any 
fixed  sense  a  bounded  linear  functional  and  indeed,  for  a  given  !  e  H, 

£(U)  need  be  defined  only  with  probability  one. 

The  preceding  discussion  of  Hilbert  spaces  has  been  principally  only 
for  orientation.  We  need  at  first  be  concerned  only  with  finite  dimensional 
subspaces  defined  as  follows:  disregard  all  but  a  finite  set  of  the  linear 
functionals,  along  with  their  linear  combinations.  We  define  the  projection 
of  the  measure  apace,  and  the  measure,  into  this  finite  dimensional  apace 
by  identifying  all  functions  u(x)  which  agree  in  the  values  taken  for  them 
by  each  of  this  finite  set  of  linear  functionals.  These  finite  dimensional 
spaces  are  euclidean  wi4b  the  inner  product  we  have  introduced.  The 
characteristic  functional  and  the  variance  of  each  of  these  finite  dimensional 
projections  of  tha  probability  measure  will  have  the  same  values  as  when  thay 
were  considered  to  be  defined  on  the  infinite  dimensional  space  and  the 
characteristic  functional  dafinad  on  tha  conjugats  specs  will  thus  be  a 
function  only  of  tha  diatanea  from  tha  origin.  That  is  to  say  that  it  will 
be  spherically  symmetric.  Zt  follows  immadiataly  that  the  n  dimensional 
measure  is  spherically  symmetric  and  muat  be  described  by  a  spherically 
symmetric  density  —  at  least  if  we  assume  it  to  be  described  by  a  density 
at  ail,  and  we  do.  Even  though  no  spheres  nor  radii  sra  dafinad  on  our 
infinite  dimensional  space  (at  least  not  with  positive  probability)  wa  may 
nonethelesa  define  spherical  symmetry  of  the  measure »  A  measure  it  spherically 
aynmetric  if  all  its  projections  into  finite  dimensional  eubspeoas  are 
spherically  aymmatric. 

Characterization  of  Spharlcally  Symmetric  Measures  on  Infinlta 
Dimensional  Spaces* 

In  each  finite  dimensional  projection  of  a  spharioally  aynmetric 
measure  the  density,  if  supposed  to  exist,  must  be  the  same  function  of 
the  distance  from  the  center  as  in  any  othar  projection  of  the  same 
dimension.  In  n  dimensions,  1st  ths  density  at  distance  r  from  the  oenter 

*We  are  indebted  to  J.  Feldman  and  R.  M.  Dudley  for  the  information  that 
this  result  concerning  spherically  symmetric  measures  in  infinite 
dimensional  spacas  is  not  new.  Zt  wes  published  in  1962  by  Uaearara,  who 
obtained  it  in  a  more  recondite  context. 
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be  pn(r  ).  Then,  considering  the  projection  of  the  measure  from  n  +  2 
dimensions  to  an  n  dimensional  subspace ,  an  easy  argument  shows  that 

p^(r2)  *  -  *  Pn+2(r2)  (3>1) 

From  this  it  follows  that  the  derivatives  of  each  of  the  p's  alternate  in 
sign.  Such  functions  are  called  completely  monotone.  There  is  a  theorem 
of  S.  Bernstein  [see  e.g..  Feller,  Th.  of  Probability,  Vol  II,  p  415] 
which  states  that  a  completely  monotone  function  p(z),  0  <_  z  <_  ■»,  with 
p(«0  ■  0  can  be  expressed  as  a  linear  aggregate  of  decreasing  exponentials 
with  positive  coefficients: 


p(z)  ■  J  e‘X*  d#(X ),  0  <  z  <  •»,  (3.2) 

o 

with  d#(X)  _<  0. 

2  2 

Setting  a  ■  r  ,  X  «  l/2o  and  re-defining  the  measure  d+(X),  we 
may  then  write 

pn<r2)  ■  $  pn0(r2)  *♦<«>  0.3) 

o 


where 

Pn(J<r2)  ■  (o/(2n))-a  e"  (3.4) 

is  the  n  dimensional  gaussian  density.  This  formula,  once  obtained  for 
any  value  of  n,  implies  the  seme  formula  for  all  lower  dimensionel 
densities,  as  is  seen  by  successive  integration  with  respect  to  each  of 
an  orthogonal  set  of  coordinates.  The  integrals  ere  all  absolutely 
convergent  and  may  ba  integrated  freely  in  any  order.  The  same  statement 
is  then  true  for  all  n.  Further  the  corresponding  statement  may  be 
asserted  expressing  the  given  measure  a  similarly  in  tense  of  the 
gaussian  measures  m0t 

m 

m  m0  d4(o)  (3.5) 
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Ai!justir.2  the  Parameters  of  the  Model 

The  proiredur?  we  are  to  follow  la  now  quite  clear.  Whatever 
may  be  the  distribution  of  the  individual  linear  functionals,  we  shall 
adjust  the  density  p^r  )  to  it  by  choosing  the  weights  dk(o)  in (3. 3).  a 
necessary  condition  is  of  course  that  the  density  be  a  completely  monotone 
function  of  r  .  Rut,  as  Peterson  has  pointed  out  above,  it  lies  very  near 
an  exponential  function  e_ar,  which,  fortunately,  satisfies  this  condition. 

We  ehall  need  only  to  be  fine  with  the  small  residue,  if  there  be  any,  and 
its  derivativss,  and  i.isiat  that  it  conform.  There  will  then  be  the  task 
of  fitting  the  retnainir  frse  element,  the  variance  of  a  linear  functional. 
Here  there  is  a  great  dial  more  freedom.  There  ie  a  functional  to  be  adjusted 
to  approximate  at  best  we  can  the  statistical  interdependence  of  the  values 
of  U(x)  at  neighboring  values  x.  (Ws  want  than  to  become  independent  at 
distant  points.)  But  this  is  just  the  sane  problem  to  be  faced  in  fitting 
a  gaussian  random  function.  For  any  £(u)  we  neied  only  go  to  the  samples  we 
wish  to  fit  and  estimate  E(£(U)  ),  or  what  is  simplar  to  tabulate,  for 
some  linear  basis  of  ths  linsar  functionals  we  estimate 

E<#,  (U)£j (U) )  from  the  sanples  for  all  pair*  i,J.  There  is  no  arbitrary 
decision  lsft  to  be  made.  It's  just  a  question  of  whather  it  works  or  notl 
or  how  well  it  workol 

Sad  to  say,  it  doesn't  work  at  all.  We  shall  see  this  without 
any  further  examination  of  samplas.  The  reason  lies  in  an  additional 
significant  difference  between  the  finite  and  infinite  dimensional  cases. 

Lack  of  Eraodicity 

We  shall  see  that  (3.5)  is,  in  a  reasonable  ssnss,  an  orthogonal' 
representation  of  the  measure.*  For  this  purpose  it  is  convenient  (and 


*For  ths  sourco  of  ths  train  of  thought  which  lsd  to  this  analysis,  ws  are 
indebted  to  Jacob  Feldman  for  a  lucid  and  provooative  briefing  on  relatively 
singular  measures,  a  briefing  which  grew  out  of  a  discussion  scsas  years  ago 
of  tha  application  of  information  theory  to  empirical  functions.  But  the 
simple  oase  with  which  alone  we  need  be  oonoemed  here  waa  known  to  us  as 
well  aa  many  other  people  long  ago.  It  appears,  for  example,  in  a  paper  of 
W.  T.  Hartih  and  X.  H.  Cameron  in  ths  lBWO's* 
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perhaps  something  equivalent  is  also  necessary)  to  introduce  a  sequence 
of  linearly  independent  bounded  linear  functionals  £^(u) ,  1=1,2,..., 
which  we  can  then  as  well  suppose  to  have  been  replaced  by  an  orthonormal 
sequence,  so  that 


EUi(uK;j(u))  =  5ij 


(0,  i  ¥  j, 
\1,  i  ■  j. 


The  question  of  when  and  in  what  sense  does  a  sequence  of  numbers 
w  ,  1=1,2,...,  represent  a  function  u  such  that 

'i  3  Vu) 

will  not  be  discussed. 

The  random  variables 

VL  =  *t(U),  1=1,2 


(5.1) 


(5.2) 


I  M  ) 


(5.3) 


are  unoorrelated  but  not  necessarily  independent.  However,  for  any  one 
of  the  gai 
variables 


of  the  gausaian  measures,  ng,  calling  the  random  function  l>0,  the  random 


".i  ■  W 

ere  unoorrelated  gausaian  variables  and  hence  independent. 

\  «&>  ■ 

we  have,  with  probability  1, 


Since 


(5.4) 


(5.6) 


lio  £  W  , 

H  ♦  •  i«i  oi 


2  =  o2. 


(5.7) 


The  measure  mg  is  not  eaaentially  altered  if  wa  trim  its  space  to  the  set 
kg  of  sequences  w^Wj,.,.  for  which  (5.7)  is  true  and  to  those  functions 
u(x)  which  give  rise  to  such  sequences.  We  restrict  our  measure,  supposed 
to  exist  and  to  be  given,  to  the  set 


J 


* -IX 


(5.8) 
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cf  fvr.it i oils  u(x).  We  do  not  disc’iss  which  subsets  of  A  have  probability 
and  how  'r  i«  defined  sir. or  th;  .  subject  sAenib  eiuisr  coo  trivial  or  too 
difficult. 

Suppose  now  that  we  test  the  distribution  of  slopes  in  A  in 
the  same  fashion  that  was  described  above.  That  is,  we  draw  a  single 
sample  function  and  measure  slopes  at  many  points  on  it.  Further, 
for  simplicity,  suppose  these  points  are  far  enough  apart  that  we  may 
ignore  statistical  dependence  of  the  slopes.  Each  sample  function  from 
A  is,  for  some  o,  taken  from  Afl.  Slopes  at  distant  points  on  it  are  then 
independent,  identical,  gaussian  variables  and  the  sample  values  of  a 
large  number  of  them  will  characterize  their  common  distribution  as  gaussian 
with  whatever  assurance  their  number  permits.  But  haven't  we  brought  this 

about  by  artificial  tampering  with  the  ensemble?  No.  He  have  only  turned 

»<) 

a  statement  true  with  probability  one  into  a  true  statement.  Devise  a 
statistical  test  for  the  normality  of  the  distribution  from  which  a  sample 
is  taken,  using  statistics  whose  distribution  is  independent  on  the  variance 
of  the  ensemble. j  The  result  of  the  test  will  (at  least  at  any  specified 
stage)  depend  on {  only  a  finite  sample.  A  finite  set  of  linear  functionals 
has  the  same  distribution  in  A  as  in  the  original  probability  measure, 
on  the  space  wei  have  called  B,  and  the  distribution  of  the  statistics  of 

i  1 

the  test  will  thus  have  the  same  distribution  in  B  as  in  A  and  aa  in  a 
gaussian  ensemble. 

In  short  then,  these  random  functions  fail  to  represent  a  random 
terrain  since  an  orthonormal  sequence  of  linear  functionals  read  off  any 
one  sample  function  have  values  distributed  like  independent  samplings 
from  a  univariate  gaussian  ensemble.  We  demand  of  our  model  of  terrain 
on  the  contrary  that  slopes  read  at  widely  separated  points  have  a  different 
distribution,  approximately  the  one  described  by  a  bilateral  exponential 
density.  More  generally,  in  order  to  make  sense  our  random  function  model 
must  have  the  ergodic  property:  independent  identical  functionals  (such 
as  slopes  at  widely  separated  points)  must  show  the  sane  distribution  whether 
read  from  a  single  sample  or  each  from  a  different  randomly  chosen  one. 
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ABSTRACT 


The  data  taken  ,  i  a  series  of  mlnslle  tests  are  often  in  the  form  of  a 
variable  of  Interest  (such  as  radial  miss  distance  from  a  given  target)  and 
several  dependent  variables  (e.g. ,  range,  temperature,  type  of  missile 
modification)  for  each  teat  made.  In  such  cases,  It  may  be  possible  to  con¬ 
struct  a  linear  statistical  model  relating  the  main  variable,  y,  to  the  others, 
through  Xy  The  coefficients  of  this  model  can  be  estimated  by  a  least 

squares  procedure. 

The  difference  between  each  measured  y  and  the  y  predicted  by  the 
linear  model  is  oalled  a  residual.  If  the  set  of  residuals  Is  normally  distri¬ 
buted,  several  well-known  tests  of  statistical  hypotheses  and  methods  of  setting 
confidence  Intervals  are  applicable.  A  procedure  for  graphically  validating 
the  normality  of  the  residuals  has  also  been  developed. 
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1.  INTRODUCTION 

In  the  summer  of  1968  the  Systems  Evaluation  Branch*  had  the  task 
of  determining  which,  If  any,  of  three  modifications  of  a  certain  missile  wan 
"beat. "  A  modification  wan  considered  "best"  if  the  average  ladlal  miss 
distance  measured  from  the  center  of  a  target  of  fixed  size  was  significantly 
less  for  the  modification  than  for  the  other  two  modifications. 

There  was  no  lack  of  data  for  this  project;  in  fact,  data  had  been 
recorded  for  over  1000  firings  of  the  missile.  For  each  firing,  the  following 
had  been  recorded:  radial  miss  di  lance  (y),  target  altitude  at  Intercept  (v<), 
range  of  the  target  at  launch  <v2) ,  range  of  the  target  at  intercept  (vs) ,  target 
closing  velocity  at  intei  cept  (v4)  ,  missile  modification  (v5),  target  type  (v6), 
and  radar  power  (v7>. 

The  data  were  sorted,  for  duplications  and  missing  values.  There  remained 
data  on  over  900  firings.  Of  these,  approximately  6  percent  were  Mod  1  firings, 

IS  percent  were  Mod  2  and  79  percent  were  Mod  3.  For  this  paper,  100  firings 
were  choaen  from  the  total;  6  of  Mod  1,  15  of  Mod  2  and  79  of  Mod  3.  Since 
the  original  data  were  classified,  the  values  were  coded  or  transformed  to 
nonstandard,  undefined  "units."  The  coded  data  are  shown  hi  Table  I. 

The  following  simple  procedure  was  considered:  divide  the  data  into 
three  groups  according  to  modification.  Calculate  the  sample  average  and 
sample  variance  of  the  radial  miss  distances  for  each  group.  Test  these  values 
for  equality  using  the  F  and  t  statistical  tests.  This  procedure  was  rejected  for 
the  following  reason:  the  testing  procedure  was  not  planned  In  advance  to  Insure 
sets  of  comparable  conditions  for  each  modification.  For  example,  moat  of  the 
firings  for  Mod  1  were  with  tho  second  target  type  (v,  ■  2) .  Thus,  if  the  above 
test  procedure  had  beek  used,  the  effect  on  the  radial  miaa  distance  of  the 
modification  would  have  been  confounded  with  the  effect  of  the  target  type.  The 
conclusion^  would  then  be  questionable  at  best. 


*  Advanced  Systems  Laboratory.  Research  and  Engineering  Directorate, 
U.  S.  Army  Missile  Command,  Redstone  Arsenal,  Alabama. 
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TABLE  I.  CODED  MISSILE  DATA,  EXAMPLE  I 


.V 

vi 

v2 

If  , 

V4 

'  6 

v* 

V- 

3.8 

1.  11 

27.  1 

24.3 

110.0 

3.0 

1.  0 

1.0 

4.7 

1.88 

27.  1 

20.  1 

108. 2 

3.0 

1.  0 

1.0 

5.0 

2.-99 

31.  3 

24. 

1H7.4 

2.0 

2.  Cl 

2.0 

5.  0 

1.24 

24.3 

20.  1 

103.7 

2.0 

1.0 

2.0 

5.  0 

1.67 

15.9 

13.  1 

127. 1 

3.0 

1.  0 

1.0 

5.0 

1.55 

29.9 

24.3 

101.0 

3.0 

1,0 

2.0 

5.6 

2.32' 

45.3 

35.5 

160.4 

3.0 

2.0 

1.0 

6.2 

6,  06 

42.  5 

27.  1 

191.0 

3.0 

2.0 

2.0 

6.8 

6.06 

36.0 

27.  1 

204.5 

3.0 

2.0 

2.0 

7.  1 

7.  )5 

46.7 

32.7 

174.6 

2.0 

2.  0 

2.0 

7.4 

1.  55 

32.7 

27.  1 

108.2 

3.0 

1.0 

1.0 

7.4 

1.33 

29.9 

'  25.7 

110.0 

3.0 

1.0 

2.0 

7.7 

1.  73 

34. 1 

25.7 

128.0  . 

1.0 

2.0 

2,0 

7.7  ■ 

3.64 

31.3 

25.7 

123.5 

3.0 

1.0 

1.0 

8.0 

'  2.34 

45.3 

35.5 

112.7 

2.0 

1.0 

2.0 

8.0 

2.32 

29.9 

21.5 

108.2 

3.0 

1.0 

2.0 

8.3 

1. 11 

31.3 

25.7 

107.3 

3.0 

1.0 

1.0 

8.3 

10.  02 

43.9 

31.3 

191.9 

3.0 

2.0 

1.0 

8.9 

2.21 

24.3 

20.  1 

123.5 

3,0 

1.0 

1.0 

9.2  . 

1. 66 

38.3 

29.9 

108.2 

2.0' 

1.0 

2.0 

9.2 

1. 88 

28.5 

20.  1 

110.0 

3.0 

1.0 

1.0 

9.2 

1.22 

28. 5‘ 

22.9 

108,2 

3.1 

1.0 

2.0 

9.5 

2.87 

31.3 

25.7 

108.2 

3.0 

1.0 

1.0 

10.1 

1.  34 

28.5 

21.5 

182.0 

3.0 

2.0 

1.0 

10.1 

1. 11 

29.9 

20.  1' 

114.5 

3.0 

1.0 

1.0 

10.1 

1.  67 

35. 5 

25,7 

188.3 

3.0 

2.0 

2.0 

10.4 

6.02 

29.9 

24.3 

123.5 

2.0 

1.0 

2.0 

10.7 

1. 88 

22.9 

17.3 

174.8 

1.0 

2.0 

2.0 

10.7 

1.66 

28.5 

24.3 

110.0 

3.0 

1.0 

2.0 

11.0 

1.55 

24.3 

20.  1 

108.2 

3.0 

1.0 

2.0 

11.0 

1. 11 

38.3 

32.7 

114.5 

3.0 

1.0 

1.0 

11.3 

1.55 

32.7 

27.1 

108.2 

3.0 

1.0 

1.0 

11.6 

2.32 

25.7 

21.5 

110.0 

3.0 

1.0 

2.0 

11.9 

2.98 

24.3 

21.5 

107.3 

3.0 

1.0 

1.0 

11.9 

1.33 

29.9 

22.9 

108.2 

3.0 

1.0 

2.0 

12.2 

1. 24 

35.5 

28.5 

108.2 

2.0 

1.0 

2.0 

12.2 

2.21 

27.1 

20.  1 

98.3 

3.0 

1.0 

i.O 

12.5 

1.  66 

28.5 

24.3 

110.9 

3.0 

1.0 

1.0 

12.8 

1.88 

32.7 

25.7 

114.5 

3.0 

1.0 

1.0 

13.1 

1.50 

31.3 

25.7 

107.3 

2.0 

1.0 

2.0 
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TABLE  I.  CODED  MISSILE  DATA,  EXAMPLE  I  (Continued) 


y 

Vl 

v: 

Vj 

13.  1 

2.98 

27.  1 

21.5 

13.4 

2.  10 

45.3 

32.7 

13.7 

1.67 

50.9 

36.9 

13.7 

2.65 

25.7 

22.9 

14.0 

2.98 

34.  1 

29.9 

14.0 

1. 11 

28.6 

21.5 

14.3 

1.33 

25.7 

21.5 

14.9 

6.06 

49.5 

29.9 

14.9 

l.  88 

17.3 

14.5 

IS.  5 

1.67 

41.  1 

28.5 

1S.S 

1.  77 

31.3 

27. 1 

16.1 

3. 42 

28.5 

18.7 

16.  1 

4. 30 

49.5 

35.5 

16.7 

1. 88 

28.5 

20.1 

17.0 

1. 56 

27.1 

22.9 

17.6 

3.64 

49.5 

32.7 

17.6 

2. 10 

25.7 

22.9 

17.9 

2.00 

25.7 

21.5 

18.2 

1.67 

28.8 

24.3 

18.5 

1.22 

28.5 

24.3 

18.8 

1. 55 

29.9 

24.3 

18.8 

1. 33 

35.5 

25. 7 

19.4 

2.65 

28.6 

18.7 

19.4 

1. 55 

28.5 

24.3 

20.0 

1. 99 

24.3 

14.8 

20.3 

1. 77 

21.5 

0.5 

20.3 

5.95 

31.3 

25.7 

20.6 

1.25 

22.9 

18.7 

21.2 

2.65 

21.5 

17.3 

21.2 

1. 33 

28.5 

25.7 

Vt 

vs 

v* 

vT 

95.6 

3.0 

1.0 

1.0 

164.0 

3.0 

2.0 

2.0 

161.3 

1.0 

2.0 

2.0 

121.7 

3.0 

1.0 

1.0 

105.5 

2.0 

1.0 

2.0 

123.5 

3.0 

1.0 

1.0 

117.2 

3.0 

1.0 

2.0 

107.3 

3.0 

2.0 

2.0 

122.6 

3.0 

1.0 

1.0 

209.0 

2.0 

2.0 

2.0 

108.2 

3.0 

1.0 

1.0 

144.2 

3.0 

1.0 

1.0 

211.7 

3.0 

2.0 

1.0 

117.2 

3.0 

1.0 

1.0 

101.0 

3.0 

1.0 

2.0 

200.9 

2.0 

2.0 

2.0 

108.2 

3.0 

1.0 

1.0 

81.2 

3.0 

1.0 

2.0 

85.7 

3.0 

1.0 

2.0 

108.2 

3.0 

1.0 

1.0 

108.2 

3.0 

1.0 

1.0 

198.2 

3.0 

2.0 

2.0 

121.7 

3.0 

1.0 

1.0 

110.0 

3.0 

1.0 

2.0 

316.1 

1.0 

2.0 

2.0 

108.2 

3.0 

1.0 

1.0 

114.5 

3.0 

1.0 

2.0 

112.7 

2.0 

1.0 

2.0 

101.0 

3.0 

1.0 

1.0 

108.2 

3.0 

1.0 

2.0 
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TABLE  I.  CODED  MISSILE  DATA,  EXAMPLE  I  (Concluded) 


y 

Vi 

v2 

Vv 

v< 

v5 

v>i 

v7 

21.8 

2.32 

31.3 

25.7 

96.5 

3.0 

1.0 

1.0 

22. 1 

5.40 

34.  1 

27.  1 

114.5 

3.0 

2.0 

1.0 

22.4 

1.  77 

31.3 

25.7 

108.2 

3.0 

1.0 

1.0 

23.0 

1. 11 

28.5 

21.5 

114.5 

3.0 

1.0 

1.0 

23.3 

2.32 

35.5 

28.5 

115.4 

3.0 

1.0 

2.0 

23.9 

1.  11 

27.  1 

18.7 

110.9 

3.  0 

1.0 

1.0 

23.9 

1.88 

20.  1 

17.3 

103.7 

3.0 

1.0 

1.0 

24.2 

:.  34 

22.9 

18.7 

120.8 

2.0 

1.0 

2.0 

24.2 

1  99 

26.7 

18.7 

114.5 

3.0 

1.0 

1.0 

24.8 

2.  98 

24.3 

18.7 

107.  3 

3.0 

1.0 

1.0 

24.6 

1. 22 

29.9 

25.7 

101.0 

3.0 

1.0 

2.0 

26.9 

1. 99 

34. 1 

25.7 

183.  8 

3.0 

2.0 

1.0 

28.4 

4.08 

22.9 

17.3 

137.0 

3.0 

1.0 

1.0 

29.3 

1. 11 

28.5 

24.3 

107.3 

3.0 

1.0 

1.0 

30.2 

2.00 

45.3 

31.3 

181. 1 

1.0 

2.0 

2.0 

30.2 

5. 07 

42.5 

28.5 

225.2 

3.0 

2.0 

1.0 

31.1 

1. 55 

27.1 

21.5 

108.2 

3.0 

1.0 

2.0 

31.4 

1.29 

34.1 

28: 6 

101.9 

2.0 

1.0 

2.0 

32.3 

1. 77 

32.7 

27.1 

108.2 

3.0 

1.0 

2.0 

34.1 

1.  33 

43.9 

29.9 

210.8 

9. 0 

2.0 

2.0 

35.0 

3.20 

43.9 

31.3 

184.7 

3.0 

2.0 

1.0 

37.1 

1. 56 

21.5 

18.7 

110.0 

3.0 

1.0 

2.0 

38.6 

1. 88 

20.1 

17.3 

101.0 

3.0 

1.0 

2.0 

41.3 

1.  88 

28.5 

24. 3 

119.0 

3.0 

1.0 

1.0 

41.6 

1.11 

46.7 

27.1 

374.6 

1.0 

2.0 

2.0 

45.8 

5.  73 

32.7 

24. 3 

141.5 

3.0 

2.0 

1.0 

48.5 

1. 99 

32.7 

28.5 

110.0 

3.0 

1.0 

2.0 

57.5 

1. 13 

56.5 

31.3 

386.3 

3.0 

2.0 

2.0 

66.5 

1.22 

24.3 

18.7 

174.8 

3.0 

2.0 

2.0 

69.6 

6.02 

27.1 

22.9 

108.2 

2.0 

1.0 

2.0 

2.  THE  LINEAR  STATISTICAL  MODEL  FOR  THIS  MISSILE 


It  was  decided  to  set  up  a  linear  statistical  model  I  1)  relating  the 
radial  miss  distance,  y,  to  functions  of  the  7  other  variables,  vt  through  v7. 
Through  engineering  considerations,  the  model  chosen  was: 


y  «  b„  +  b jX,  +  bjXj  +  bjxs  +  b4x4  +  bsxs  +  b*^  +  bTx7  +  b(1x6  +  b,x, 
+  b^x^t  e 

x  | »  v  j  ■  target  altitude  at  intercept 
x2  ■  v2  ■  range  of  target  at  launch 
xs  -  *  vj| 

x4  ■  vs  *  ran^e  of  target  at  intercept 
x5  -  xj  =»  v] 


x4»v4a  target  closing  velocity  at  intercept 


xt0« 


-0. 5  if  Mod  1 
0.5  if  Mod  2 
0.0  if  Mod  3 
0.  0  if  Mod  1 
-0.5  if  Mod  2 
0. 5  if  Mod  3 
(-0.5  if  target  type  1 
\  0. 5  if  target  type  2 

1-0. 5  If  low  intensity  radar 
0. 5  if  high  intensity  radar 


e  ■  random  error 


3.  GENERAL  LINEAR  STATISTICAL  MODELS 

Frequently  the  results  of  experiments  or  measurements  are  given 
as  a  set  of  independent  variables  and  as  associated  result  or  dependent 
variable.  The  data  discussed  above  provides  ono  example.  As  another  example, 
the  velocity  of  the  vehicle  could  be  measured  at  various  time  points. 

The  result  or  observation,  y,  is  oonsidered  as  a  function  of  the 
independent  variables,  v„  v,,  . . .  ,  v  ,  and  random  noise  e  and  written: 
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y  =  y  (v„  vj,  ...  vm,  e)  . 

The  observation  noise  or  measurement  noise  e  is  a  result  of  the  inaccuracy  oi 
the  measuring  devices  and  of  variables  which  arc  not  included  in  the  model  but 
which  do  affect  the  observation.  If  the  model  is  correct,  c  is  the  random 
fluctuation  of  y  for  the  fixed  values  of  v  , through  v  . 

The  moat  convenient  and  frequently  used  model  is  the  linear  statistical 

model: 

v*b0+b1x,+  b,xj+  ...  +bkxk+ e  . 

Here  the  x^’s  are  functions  of  the  basic  variables  vJt  e.  g. ,  x,=  v,,  x?  *  v^, 

Xj  *  v2,  x4  =  vt  v2.  On'  restriction  on  the  Xj's  is  that  they  be  linearly 

independent;  e. g  ,  if  x, a  v,  and  x2  =  Vj,  then  xs  cannot  be  set  to  (v,  +  v2) .  The 
other  restriction  is  that  the  x.'s  be  known  or  measured  without  error.  (Both 
l  1 

restrictions  can  be  relaxed  in  more  advanced  work. )  The  model  ia  termed 
’’linear"  because  it  is  linear  in  the  coefficients  b^.  The  b^'s  are  considered  to 

be  fixed  but  unknown  and  must  be  estimated  from  the  data. 

It  should  be  noted  thst  this  is  not  the  only  statistical  model  possible  and 
may  not  apply  In  some  oases.  However,  It  oan  be  used  suooesefully  in  s  large 
number  of  situations  and  it  does  possess  manipulative  ease.  The  model  should 
be  constructed  from  physical  and  engineering  considerations.  As  will  be 
aeen  later,  statistical  tests  can  be  used  to  determine  which  terms  can  be 
dropped  from  the  model  without  seriously  affecting  the  accuracy;  however, 
they  give  no  indication  of  which  new  terms  should  be  added  to  the  model. 

The  results  on  the  missile  disouaaed  above,  hereafter  known  as 
Example  I,  are  of  oonoern  here.  However,  in  order  to  illustrate  the  method 
with  a  small,  uncomplicated  oase,  a  simple  example  (Example  n)  was 
concocted.  The  calculations  of  Example  H  oan  be  done  by  hand  In  a  "reasonable" 
(compared  to  Example  I)  length  of  time. 

In  Example  II,  the  amount  of  catalyst  added  to  oach  of  two  vats  in  a 
chemical  plant  was  varied  from  0  to  5  units .  The  resulting  yields  are  listed 
In  Table  H. 
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TABLE  H.  CHEMICAL  YIELD,  EXAMPLE  II 


1  . . . 

Vl 

Amount  of 
Catalyst 

_  »  _ J 

Yield  for 

Vat  1 

Yield  for 
Vat  2 

0 

8.81 

7.02 

1 

10.  00 

10.02 

2 

13. 25 

3 

14.51 

4 

11.36 

5 

8.58 

mKBm 

The  yields  are  plotted  in  Figure  1  u  functions  of  the  amount  of  catalyst. 


The  plot  suggests  that  an  appropriate  model  would  be  a' second  degree  polynomial: 


y  ■  b0+  btVj+  bj  v\+  e 
-  b0  +  bj  Xj+  bj  x2+  e  , 

* 

where 

x,-  v„  x,-  . 
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Without  loss  of  generality,  it  can  be  assumed  that  the  noise  e  has  zero 
mean  (if  not,  the  mean,  E  (e) ,  could  be  included  in  the  term  b0  so  that  the 
iCuefiiieu  noise  e*  =  e-E  (e)  has  zero  mean]. 


If  a  total  of  n  observations  are  taken,  then  the  model  can  be  written  as: 
y,-b,  x,+  bIxljJtb,«8J*  ...  +bk*kj  +  »)■  1"  1 . - 


tb 


where  x„  -  1  and  x  is  the  value  of  x.  for  the  j  data  point.  To  shorten  the 

M I  * 

above  equations,  the  following  vectors  and  matrix  are  defined: 


bi)\ 

b| 

b2 


e» 


en 

•t 


fl  xui  X2il  ...  xk  l 
1  XU  x2i2  ...  xk>2 


-X* 


1  '  X.  ■  •  i  X 

l#n  2,n  k,n 


The  above  equation  becomes 


Xb+  e 


4.  ESTIMATION  OT  THE  COEFFICIENTS 

A  linear  statistical  model  has  been  postulated  In  Section  2.  In 
addition,  the  noise  e  is  assumed  to  have  zero  mean  and  covariance  matrix  *  <r*I, 
where  I  is  the  Identity  matrix  and  a2  is  a  constant  that  may  be  unknown.  That  is, 
for  1,  J  ■  1 . n,  E(e^  -  0,  var  (e^  ■  or1  and  oov  (s^,  e^  »  0  if  i  *  J.  If 

this  assumption  is  not  mat,  the  proper  transformation  of  variables,  in  most 
cases,  will  reduoe  the  model  to  one  in  which  the  assumption  does  hold. 

A  method  must  be  found  for  determining  b,  the  estimate  of  the  coeffi¬ 
cients  b.  There  is  ueually  s  loss  incurred  when  the  estimate  b  is  not  the  true 
valuo  b!  Usually,  the  further  6  lies  from  the  true  value  b,  the  greater  the  loss 
beoomes.  Since  the  values  of  X  and  y  are  given,  it  is  desirable  to  choose  6 
so  that  the  predicted  value  of  £■  xfe  will  be  close,  in  some  sense, 
actual  observation  vector  £.  A  convenient  way  of  doing  this  is  to  choose  b  so 
that  the  quadratic  loss 


445 
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T  T 

The  quantities  X  £  and  X  X  are  calculated  to  be 

/  121. 86\  _  /  12.0  30.0  110. 0\ 

Xj£“  (  302.23  j,  X  X  =  I  30.0  110.0  450.0  ] 

\1035.99/  \110.0  450.0  1958.0/ 

T 

The  inverse  of  X  X  is 

/  0.4107  -0.2948  0.0446\ 

(XTX)"‘?  (-0.2946  0.3634  -0.0640  1 

\  0.0446  -0.0670  0. 0/134  /  . 

The  estimate  of  b  is 


The  prediction  equation  for  y  is  thus 

y-  7,249 +4.549  xt-  0.924  x, 
y-  7.249  +  4i  549  Vj-  0.924  . 

Listed  below  are  £,  £,  and  the  error  in  the  prediction  of  &  £-£. 


i 

£-x6  j 

A 

x-i 

—  '  — . 

-  ■  — 

r  " 

8.81 

7.249 

1.561 

10.00 

10.874 

-0. 874 

13.25 

12.652 

» 

0.598 

14.51 

12.582 

1.927 

111  36 

10.667 

0. 693 

8.58 

6.904 

i  v 

1.676 

7.02 

7.249 

-0.229 

10. 02 

10. 874 

-0.854 

10.15 

12. 652 

-2.502 

13.43 

, 

12.582 

0.848 

10.40 

10. 667 

-0. 267 

4.33 

6.904 

-2.574 

Plotted  in  Figure  2  are  the  data  points  and  the  prediction  equation. 
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Notice  that  no  mention  haa  been  made  of  the  probability  distribution  of 
the  measurement  noise  e  except  that  the  covariance  matrix  is  ol I  and  ,the  mean 
is  the  *ero  vector.  Thus,  the  formula  for  the  least  squares  estimator  is  free 
of  the  distribution  of  e.  Also,  no  matter  what  the  distribution  of  e,  if  E  (e)  <*  0 
and  cov  (e)  *  cr*I,  then 

E  ft)  -B  (Xb  +  e)  =  Xb  f 

COV  ft )  =  COV  (X  b  +  e)  a  ffZI  . 

E  (b)  -E  ((XTX]-1  XTj)  =  [XTX]-'XTE(i)ab 
cov  (b)  -  |XTX]^>XT  cov 2.X  [XTX]  -  os  [XTXJ-‘  . 


Thus,  no  matter  what  the  distribution,  the  least  squares  estimator  is  unbiased 
*  A  T 

i  E  (£)  «  b]  and  has  covariance  matrix  cov  (b)  *  o*  (X  X)"1. 


An  appealing  estimate  of  the  variance  <7*  is  the  "average"  loss.  After 
the  coefficient  vector  b  has  been  estimated  by  b,  the  predicted  value  of  the 

dependent  variable  at  the  Jth  point  is  y^  *  x^b.  The  difference  between  the  actual 

or  measured  value  of  y^  and  the  predicted  is  called  the  residual,  r^  -  ^  - 

The  sum  of  the  squares  of  the  residuals  Is  called  the  sum  of  squares  for 


error  fSSE )  and  can  be  shown  to  equal: 
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>¥  ■*< 


n  n  i 

SSE  ~  7,  r?  -  7  /y.  -x  bV=  (V-Xb)T  ty-Xb)  . 
j  =  1  J  J-1\  J  J  7  '  ~  '  ~ 


One  estin!ate  of  the  variance  a1  is  then 


How  well  sJ  estimates  crJ  depends  upon  the  forms  of  the  probability  distribution 
of  the  noise  e. 


in  Example  It, 


SSL-  •-  25.  OL 


a?  =•  2.76.. 


The  covariance  matrix  of  h  la 


(XTX)"‘  a1  and  i>  estimated  by 


j  1.14 

-0.82 

0.12 

(-0.82 

i.oi 

-0.19 

\  0.12 

-0.19 

0.037 

! 


\ 

5.  TWO  TYPES  OF  INDEPENDENT  VARIABLES  -  QUANTITATIVE  AND 
QUALITATIVE 

/.. 

For  the  missile  model  (Example  I),  y  is  the  dependent  variahle,  v, 
through  v7  are  the  basic  independent  variables;  x ,  through  x]0  are  the  expanded  / 
variables.  The  expanded  variables  xj  through  x«  are  quantitative  variables  and  / 
x7  through  xt0  are  qualitative  variables. 


A  quantitative  variable  Is  one  to  which  such  units  as  meters,  degrees, 
and  pounds  can  be  attached.  The  quantitative  variables  include  velocity,  time, 
angle  meaaurement,  distance  and  amount. 


The  other  kind  of  variable  is  the  assigned  or  qualitative  variable  which 
represents  such  things  as  missile  modification,  type  of  stimuli,  which  of  several 
measuring  devices  were  used  to  obtain  the  data,  etc.  These  variables  must  be 
assigned  values  and  cannot  logically  be  given  units.  Certain  conventions  for  the 
assigning  of  values  have  been  set  up  for  this  paper. 
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The  estimate  b  la  thus 


7.249\ 


The  prediction  equation  is 

/  ’ 

y  =  7.249  +  4.549  Xj  -0.924  x2  -1.860  xa  . 

The  estimate  of  the  variance  is 

8s  -  1.83  . 

,  «•  A 

The  covariance  matri ::  and  the  correlation  matrix  of  b  arc  given  below: 


0.0  \ 
0.0  X 

0.0  I 

0.61  / 

o.oo\ 
0.00  1 
0. 00  / 
1.00/  . 


y  *  8, 179  +  4. 549  vt  -0. 924  v^  if  vat  1  is  iwed 

y  »  6;  319  +  4. 549  vt  -0. 924  if  vat  2  is  used. 

Thus,  the  difference  between  the  predicted  yields  from  vat  1  and  vat  2  with  the 
same  amount  of  catalyst  is  estimated  as  y  .  -  y  .  »  fis  =  -i.  860.  The  two 

curves  are  plotted  in  Figure  3.  v  v 

This  fit  may  be  compared  with  the  preceding  fit  without  the  term  for 
vat  differences. 

In  the  previous  example,  there  were  two  vats  used  and  the  values  of  -0. 5 
and  0. 5  were  rather  arbitrarily  assigned  to  represent  the  vat  used.  It  is 
noticeable  that  the  qualitative  variable  occupies  one  place  in  the  model  and  one 
column  in  the  X  matrix;  this  corresponds  to  the  one  difference  between  two 
factors. 


cov  (b)  = 


cor  (b)  a 


/  0.75 

-0.54 

0.082 

[-0.54 

0.66 

-0. 12 

l  0.082 

-0. 12 

0.24 

\0.0 

0.0 

o.o 

/l.OO 

-0.77 

0.65 

/ -0.77 

1.00 

-0.96 

l  0.65 

-0.96 

1.00 

\  0.00 

0.00 

0.00 

In  this  case,  two  curves  are  predicted 
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T  YIELD 


0  1  1  J  4  S 

V,  CATALYST 


FIGUUK  3.  PREDICTED  YIELD  FOR  THE  EXPANDED  MODEL,  EXAMPLE  D 


In  Example  I,  there  were  three  modifications  of  the  missile.  There  are 
two  linearly  independent  differences  among  the  three  effects,  A,  B,  and  C  of  the 
modifications.  Thus,  one  could  choose  B-A  and  (p-B;  in  this  case,  C-A  is  a 
linear  combination  of  the  others,  C-A  *  C-B  +  B-A.  Another  choice  of  linearly 
independent  differences  is  B-A  and  2C-B-A.  In  this  case,  two  terms  are  added 
to  the  model  and  two  vectors  are  added  to  the  matrix. 

,  In  tills  work  the  following  values  were  assigned  to  the  expanded!  variables 
for  the  modifications :  i 


Modification 

x7 

iX8 

1 

-0.5 

0.0 

2 

0.5 

-0.5 

3 

0.0 

0.5 

I  f  there  are  four  types  for  a  qualitative  variable,  then  there  are  three 
independent  vectors.  They  could  be  assigned  the  following  values: 


Type 

*1 

x2 

*S 

1 

-0.5 

0.0 

.0.0 

2 

0.5 

-0.5 

0.0 

3 

0.0 

0.5 

-o\  5 

4 

0.0 

0.0 

0.5 

The  name  pattern  is  followed  for  other  numbers  of  types. 
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6.  THE  ESTIMATED  COEFFICIENTS  TOR  THE  MISSILE  MODEL 


tHo  Hfitft  for  missils 


/V  vo  w  nl  ft  1  \ 


iiawo  inmif  Info  fV n 


Generalized  Least  Squares  Fit  (GELSF)  digital  computer  program  |2J,  which 
was  written  to  do  the  above  calculations.  The  results  are  shown  in  Tables  III 
through  VI.  The  predicted  model  is 


y  =  24. 103+  0.086x,  -  1.496x2  +  0.020xs  +  0.866x<  -  0.  016x5  +  0.  087x* 
+  6.  719xt  +  6.  413x8  -  2.061x9  +  3.753x10. 


a.  ,The  Advantages  of  Normal  Noise 

In  the  special  case  of  Gaussian  or  normal  noise,  the  least 
squares  estimator  is  a.so  the  maximum  likelihood  estimator.  The  likelihood  func¬ 
tion  is  the  joint  probability  density  of  the  observations.  Assuming  X  is  known 
perfectly,  b  is  fixed  but  unknown,  the  noise  e  Gaussian  with  mean  0  and  covari¬ 
ance  matrix  <r2l,  the  observations  will  be  Gaussian  with  mean  Xb  and  covariance 
matrix  o2I.  Thus,  the  likelihood  function  is  given  by 

n 

Lh.  =  2  exp  (£-Xb)T  (£-  X  b) 

If  the  derivative  of  the  likelihood  function  with  respect  to  b  is  set  to  zero,  the 
value  b  which  maximizes  the  likelihood  is 

b«  <XTX)-'XTe  . 

This  is  Identical  to  the  least  squares  estimator.  If,  however,  the  distribution 
of  the  noise  is  other  than  Gaussian,  the  likelihood  function  and,  thus,  the 
maximum  likelihood  estimator,  may  be  different  from  the  least  squares 
estimator. 


Furthermore,  if  the  covariance  matrix  is  of  the  form  Q  ■  o2I  and  the 

noise  is  Gaussian,  then  it  can  be  shown  that  the  ratio  ^  ~  —  =1  jji "  2Lal 

a2 

has  a  Chi-square  distribution  with  (n-k-1)  degrees  of  freedom  where  n  is  the 
number  of  data  points  or  observations  and  k  is  the  number  of  x 's  in  the  model, 
*  T  »  1 

and  thus,  ~  X.^  *  %  “  X  is  an  unblued  estimator  of  cr2.  This  is  the  s2 


discussed  in  Section  4. 
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TABLE  HI.  THE  ESTIMATED  COEFFICIENTS,  EXAMPLE  I 
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TABLE  V.  THE  ESTIMATED  COVARIANCE  MATRIX,  EXAMPLE  I 
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TABLE  VI.  THE  ESTIMATED  CORRELATION  MATRIX,  EXAMPLE  I 
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Farther,  the  assumption  of  normally  distributed  noise  is  used  in  the 
statistical  teats  and  confidence  Interval*  to  be  discussed  In  Section  10. 

The  data  for  Example  !1  was  generated  under  the  assumption  oi  normal 
noise,  liuwtrvtrj  ,  «r  in  a  quesitoii  u»  be  asked  uboui  ihe  missile  data  of 
Example  I 


7  VESTING  THE  MISSILE  DATA  FOR  NORMAL  NOISE 


Before  conclusions  based  upon  the  assumption  of  normally  distributed 
noise  can  be  drawn  for  Example  I,  a  test  for  normality  must  be  made.  The 
residuals  factual  y  ■■  predicted  y)  estimate  the  error  e  and.  thus,  should  be 
tested  for  normality. 

It  was  decided  to  use  the  method  of  normal  probability  paper  and 
control  bands  to  test  the  residuals  for  normality.  The  Testing  for  Normality  by 
Control  Bands  (TEN  COB)  digital  computer  program  was  used. 


8.  NORMAL  PROBABILITY  PAPER 


The  construction  of  normal  probability  paper  is  similar  to  that  of 
logarithmic  paper.  Assume  that  the  random  variable  r  haa  a  normal  (Gaussian) 
distribution  with  mean  p  and  variance  crs.  Then  the  reduced  variate  v  =  (r  -  p  )/a 
haa  a  standard  normal  distribution,  i.e. ,  v  has  mean  0  and  variance  1.  If  r  is 
plotted  on  a  horizontal  linear  scale  and  v  1b  plotted  on  a  vertical  linear  scale, . 
the  straight  line  r  a  cr  v  +  p  will  result  (Figure  4 ) . 


Since  v  is  a  standard  normal  random  variable,  the  cumulative  distribu¬ 
tion  function  of  v  is  given  by 


F<v»  / 


-¥ 

c-  dt 


On  a  second  vertical  scale  the  distribution  function  F  (v)  is  plotted  (Figure  5). 
The  values  of  F  (v)  -  0  and  F  (v)  =  1  never  appear  on  the  scale,  since  these 
correspond  to  values  of  v  =  -  «  and  v  -  •*>,  respectively.  Ifthelines  aredrawn  forthe 
function  F  (v)  instead  of  v,  a  nonlinear  vertical  scale  is  shown  (Figure  6). 


a.  Plotting  of  Points  on  Probability  Paper 

It  is  desired  to  test  whether  the  underlying  probability  distribu¬ 
tion  of  the  ii  residuals  is  normal  or  can  be  approximated  by  a  normal  distribution. 
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FIGURE  4.  THE  REDUCED  VARIATE 


FIGURE  6.  EXAMPLE  OF  NORMAL  PROBABILITY  PAPER 


If  the  distribution  is  indeed  normal,  the  residuals,  when  plotted  against  F  (v) 

In  the  manner  discussed  below,  should  approximate  a  straight  line,  r  »  y  +  v/6. 
The  deviations  from  tho  line  are  caused  by  the  finite  random  character  of  the 
sample  of  size  n  and  Buch  errors  as  round-off. 

Let  r,t  r2 . rn  be  the  n  reeidualB  arranged  in  ascending  order. 

Several  methods  of  plotting  this  sec.uenct;  of  numbers  against  F  (v)  are  considered 
by  Gumbel  (3) .  Tho  best  method  is  that  of  plotting  r^  against  J/  (n  +  1).  This 

method  is  distribution  free  and  all  observations  can  be  plotted.  Further,  the 
plotting  positions  are  simple  to  calculate. 

Listed  In  Table  VT1  are  the  100  residuals  arranged  In  ascending  order. 

The  corresponding  values  of  F  (v)  =  j/  {n  +  1)  and  v  are  also  listed. 
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-11.932 

10 

0.0990 

-1.255 

-11.608 

11 

0. 3089 

-1.232 

-11.548 

12 

0. 1188 

-1. 180 

-11.071 

13 

0. 1287 

-1.  132 

-10.949 

14 

0. 1386 

-1.086 

-10.748 

15 

0. 1485 

-1.042 

-10.701 

16 

-1.001 

-10.677 

17 

0.  1683 

-0.960 

-10.487 

IS 

0. 1782 

-0. 922 

-9.860 

19 

0.  1881 

-0.884 

-9.328 

20 

0.  1980 

-0.848 

-9. 088 

21 

0.2079 

-0.813 

-9.008 

22 

0.2178 

-0.779 

-8.835 

23 

0.2277 

-0. 746 

-8.666 

24 

0.2376 

-0. 714 

-7.695 

25 

0.2475 

-0.682 

-7.629 

26 

0.2574 

-0.651 

-7.568 

27 

0.  2673 

-7.251 

2;- 

0.  2772 

-0. 59 1 

-7.065 

J 

o. 2*7! 

-(I.  ~)f)  1 

-6.615 

30 

0. 2970 

-0.532 
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rAHLE  VII  HESII)I  AL.S  OF  EXAMPLE  I  WITH  CORRESPONDING 
REDUCED  V AP' ATE  (Continued) 


r . 

i 

F  (v)  =  i/101 

V 

Reduced 

The  Residuals 

j 

Distribution 

Variate 

in  Ascending 

Function  Plot¬ 

Plotting 

Order 

The  Rank 

ting  Positions 

Positions 

-6.49a 

31 

0. 3069 

-0.504 

-6.351 

32 

0.3168 

-0.  476 

-6. 185 

33 

0.  3267 

-0.448 

-5.644 

34 

0.  3366 

-0.421 

-5.636 

35 

0.3465 

-0.394 

-5.319 

36 

0.3564 

-0.367 

-4.996 

37 

0. 3663 

-0.341 

-4.921 

38 

0. 3762 

-0.315 

-4.733 

39 

0.3861 

-0.289 

-4.729 

40 

0.3960 

-0.263 

-4. 103 

41 

0.4059 

-0.238 

-3.852 

42 

0.4158 

-0.212 

-3.833 

43 

0.4257 

-0. 182 

-3.593 

44 

0.4356 

-0. 162 

-3.563 

45 

0.4455 

-0. 136 

-3.373 

46 

0,4554 

-0. 112 

-3. 156 

47 

0.4653 

-0.086 

-2.919 

48 

0.4752 

-0. 062 

-2.874 

49 

0.4851 

-0.  037 

-2.775 

50 

0.4950 

-0.012 

-2.548 

51 

0.5050 

0.012 

-2.464 

52 

0.5149 

0.036 

-2.  188 

53 

0.5248 

0.062 

-1.814 

54 

0.5347 

0.086 

•  -1.039 

55 

0.5446 

0. 112 

-0.617 

56 

0.5545 

0.  136 

-0.374 

57 

0.5644 

0.  162 

-0.  122 

58 

0.5743 

0.  187 

0.  120 

59 

0.5842 

0.212 

0.276 

60 

0.5941 

0.238 
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TABLE  VII  HEHIDUAI.S  OF  EXAMPLE  I  WITH  COHilE.SI'ONDIN'C, 
REDI'CFI)  VAJtiATE  lOcUinurd) 


r 

j 

F  (v)  =  j/101 

Reduced 

The  Residuals 

1 

Distribution 

Variate 

in  Ascending 

Function  Plot¬ 

Plotting 

Order 

The  Rank 

ting  Positions 

Positions 

0.279 

61 

0.6040 

0.263 

0.592 

62 

0.6139 

0.289 

0.616 

63 

0.6238 

0.315 

0.690 

64 

0.6337 

0.341 

1.535 

65 

0. 0-‘ 36 

0.  367 

1.941 

66 

0.6535 

0.  394 

2.067 

67 

0.6634 

0  421 

2.420 

68 

0.6733 

0.  448 

2.680 

69 

0.6832 

0.476 

3.507 

70 

0.6931 

0.504 

3.649 

71 

0.7030 

0.532 

5.142 

72 

0.7129 

0.561 

6.212 

73 

0.7228 

0.591 

6.216 

74 

0.7327 

0.621 

6.254 

75 

0.7426 

0.651 

6.630 

76 

0.7525 

0.  682 

6.901 

77 

0. 7624 

0.714 

6.943 

7a 

0. 7723 

0.746 

7.071 

79 

0. 7822 

0.779 

7.351 

80 

0.7921 

0.813 

7. 7 1C 

81 

0.8020 

0.848 

7.864 

82 

0.8119 

0.884 

7.940 

83 

0.8218 

0.922 

8.111 

84 

0.8317 

0.  960 

9.085 

85 

0.8416 

1.001 

9.664 

86 

0.8515 

1.042 

10.505 

87 

0. 96 11 

1.  086 

12.326 

88 

0.8713 

1.  132 

13.307 

89 

0.8812 

1.  180 

13.396 

90 

0.8911 

1.232 
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TABLE  VII.  RESIDUALS  OF  EXAMPLE  I  WITH  CORRESPONDING 
REDUCED  VARIATE  (Concluded) 


r  v 

) 


The  Residuals 
in  Ascending 
Order 

1 

The  Rank 

F  (v)  -  j/101 
Distribution 
Function  Plot¬ 
ting  Positions 

Reduced 

Variate 

Plotting 

Positions 

14.072 

91 

0. 9010 

1. 285 

14. 830 

92 

0.9109 

1. 347 

14.912 

33 

0. 9208 

1.410 

16.841 

94 

0.9307 

1.480 

17.  372 

95 

0. 9406 

1.559 

24.229 

96 

0.9505 

1. 649 

29.745 

97 

0.9604 

1. 755 

30.698 

98 

0.9703 

1.885 

40. 946 

99 

0.9802 

2.058 

51.625 

100 

0.9901 

2.329 

Figure  7  shows  the  points  plotted  on  normal  probability  paper  by  a 
modified  version  of  the  TEN  COB  program.  The  horizontal  scale  is  the 
r-scale;  the  horizontal  line  defining  the  grid  extends  from  -37.665  to  69.981 
for  the  case.  The  vertical  scale  is  the  reduced  variate  or  v-scale;  the  vertical 
line  defining  the  grid  extends  from  v  *  -4. 0  to  v  =  4. 0. 


b.  Fitting  the  Straight  Line 

If  the  scatter  of  the  plotted  points  is  very  small,  the  best  fitting 
straight  line  can  be  found  by  lining  up  a  ruler  through  the  points.  However,  in 
many  canes  this  Is  not  satisfactory.  It  is  then  necessary  to  estimate  6  and  y, 
the  two  parameters  of  the  straight  line; 

r  =  y  +  £ ;  v  =  6  (r  -  y )  . 

For  the  estimation  of  these  parameters,  the  classical  method  of  least  squares 
will  be  employed. 

In  the  method  of  least  squares ,  either  the  stun  of  Bquares  of  the  horizon¬ 
tal  deviations  of  the  points  from  the  estimated  straight  line, 
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2 


n 

y 

i—j 

i 


or  that  of  tilt-  vertical  distances. 


n 

y 

it  1 


V;  ~  ^  (  ri 


is  minimized.  Here  a,  and  a2  represent  the  two  estimates  of  <5,  and  g(  and  g2 
represent  the  two  estimates  of  y.  If  the  partial  derivatives  of  the  first  sum  with  \ 
respect  to  a,  and  g,  are  set  to  zero,  the  results  are 


and 


or 


n 


»-  1 


0 


r-(fi-“v=0, 

at 


rv  -  Kt  v  -  —  v*  «  0  , 

ai 


where 


n  n 

-  1  y  -  1  V 

^  ri’  V=n  ^  V 

la  1  i-  1 


I  I  II 

-  1  V  2  1  V  2 

rv  -  \  r.  v  ,  v*  -  -  A  v,  . 
n  i  i  n  ,‘-y,  i 


i-  1  '  '  "  i=  1 

For  a  normal  distribution,  v-  0.  Thus,  the  solution  is 
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j-  JZ_ 

V 


*1  1‘ 


^  h  V  i  ' ' 


~  — 

V*  -  V  i  V 


If  the  sum  of  squares  of  the  vertical  aeviations  is  to  he  minimized,  the 
parti als  with  respect  to  g2  and  a:  arc  set  to  o, 


and 


or 


where 


! .  iv  -  a,  It  -  s.-\ 

i‘  i  L 1  '■  !\ 


0 


n 

i  1  /  V 

y  v 

vi  ‘  *  (ri  “  g2) 

=  0 


v-a2r+a2ftss0 
rv-a?rz  +  aig2r  =  0, 

7  r  2 

n  ,Lj,  i 


With  v  s  0,  the  solution  is 


i=  1 


1  r2  - 


a,  Fv 
82  =  r  • 

The  estimate  1/a^  is  altered  slightly  to 

s  2 
_1_  _r_ 

a2  ”  rv  ’ 
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where  s^2  is  the  sample  variance, 


In  order,  to  combine  the  two  estimates  of  1/6  and  eliminate  the  crose 
product  the  geometric  mean  of  1/a ;  and  l/aj  is  found.  Thus,  the  two  combined 
estimates  are 

l_  M  J_  jr 

a  v  a,  a,  ~  s 

,  1  v 

g  -  r  . 

Th**se  eptinates  require  the  calculation  of  only  F  and  s^  from  the  sample. 

The  value  of  s  depends  only  upon  the  number  of  points  n.  As  the  sample  size  n 

increases,  s  approaches  1  and  the  estimate  r  and  s  approach  the  true  values 

M  and  o;  thus,  the  equations!  the  straight  line  approaches  r  ■  ji  +  a  v,  discussed 
in  Section  9.  J 

The  estimated  straight  line, 


s 

_  r 

r  =  r  +  —  v  , 
s 

v 

is  plotted  on  the  same  paper  as  the  points.  If  the  points  lie  close  to  the  esti¬ 
mated  line,  the  distribution  is  considered  to  be  approximately  normal.  If  the 
scatter  Is  toe  great  with  respect  to  the  line,  the  distribution  is  considered  non¬ 
normal.  To  determine  whether  the  Bcatter  is  too  great/ control  bands  arc  placed 
around  the  line. 

For  Example  I,  the  values  range  from  -19. 799  to  51.625.  The  end-points 
of  the  graph  are  -.17.655  and  69.481.  The  average  residual,  r  =  0.00286;  the 
estimate  of  l,  is  l/a  -  12.353.  In  Figure  8  are  shown  the  points  of  Example  I 
plotted  along  uith  the  estimated  straight  line,  r  -  0.002  86+  12.35  3  v. 


c,  Control  Banda 


The  j  largest  observation  r  in  a  sample  of  size  n  is  called  the 
th  J 

j  order  statistic.  Each  observation  was  drawn  from  a  population  with  density 
function  f  and  distribution  function  F.  In  this  caae,  the  Initial  distribution  is 

assumed  to  be  normal.  The  j  order  statistic  r.  haB  a  derived  density  f^r. j 

that  depends  upon  the  initial  distribution  F  and  upon  the  values  of  j  and  n .  It 
car  be  shown  that  this  derived  densit>  function  is 

'»  (rj)  *  MK.1-1H  ^ ’  [' ri)  [ : '  -  F '  (r,)  • 

«.  J 

Substituting  the  equati'  ns  for  a  normal  variate  for  F^r.  ^  and  ono  ot)t-aira 

,  fs  exp  -  73  (t  ~  *0*  \*  1 

'"('i)’  (“-ilWi-ll!  H„  * 


r.  1  n-j 

-«  \/2¥"(r 

•  7T“expf'^(rj"M)i 

sTn  cr  L  '  J  '  J 

This  complicated  form  does  not  reduce  to  anything  more  reasonable  and  Is 
difficult  to  manipulate  and  calculate.  Thus,  asymptotic  distributions  of  the  order 
statistics  are  used. 

As  n  becomes  larger,  either  j  will  increase  with  n  so  that  i/n  remains 
approximately  constant  or  ]  will  remain  constant  so  that  j/n  decreases.  In  the 
th  th 

former  case,  the  j  value  r^  le  called  the  j  central  value;  in  the  Becond  case, 

the  values  r.  and  n-i  +  1  are  called  extreme  values. 

J 

It  can  be  shown  that  as  n  increases,  the  distribution  of  the  jt81  central 
value,  r  ,  becomes  asymptotically  normal  with  mean  and  variance 
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E  r.  =•  r* 
J  J 


•  >  ■  !r(ri)]  [1-F('r,^ 

var  /r.\  -  j.~  - 1 “ - - ^ 


"f(D 


where  r*is  the  solution  of 
J 


tr0*- 


The  asymptotic  distribution  is  used  within  the  interval  0. 15  r~  F  s  0. b5,  although 
its  accuracy  within  tlis  interval  depends  upon  the  sample  ni/.o. 


For  the  reduce  ’  variate  v. 


var 


'i-“(rrv) 

(vj)-oi  v“(rj) 


,  the  variance  is  given  by 


and  is  Independent  of  the  parameters  a  and  g.  The  product  Jn  •  var  ^v^ 

is  independent  of  the  sample  size  and  is  a  function  of  the  initial  distribution  only  . 
A  chart  of  this  product  for  several  values  of  F,  assuming  that  the  initial 
standard  deviation  a  -  1,  is  shown  In  Table  vm. 


/ 


TABLE  VIII.  REDUCED  STANDARD  ERRORS  FOR  UNIT 
STANDARD  DEVIATION 


Probability,  F 

Jn  ■  yU  (Vj) 

0.15 

1.532 

0.20 

1.429 

0.25 

1.363 

0.30 

1.318 

0.35 

1.288 

0.40 

1.268 

0.45 

1.257 

0.50 

1.253 

0.55 

1.257 

0.60 

1.268 

0.65 

1.288 

0.70 

1.318 

0.75 

1.363 

0.80 

1.429 

0.85 

1.532 
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Because  of  the  relationship  r  =  —  v  +  y,  the  standard  deviations  are 


r.  'mod  1>v  s.d. 


-  s.d.  ^y^/a*.  This  can  be  written  in  the  form 
fn  ■  var  (v.\ 

Thus,  the  standard  error  of  the  value,  m,  can  be  estimated  by  multiplying 

the  value  Jn  •  var  ^v.^  by  x  ij.  For  evarnple,  for  the  data  given  in 

Section  8.  a,  n  1(  0  and  -  =  12.  353,  so  s.d.  |r20)  =  1.429  (12.  353)/ 10  =  1.756. 

a  ‘ 

this  standard  erroi  is  used  in  the  construction  of  control  bands. 

Assume  that  a  and  g  have  been  estimated  and  that  the  observations  and 
the  estimated  straight  line 


1 

r  =  g  +  -  v 
a 


have  been  plotted.  For  each  probability  value  listed  in  Table  VIII,  the  estimate 
ot  r  is  found  by  intercepting  the  estimated  line  with  a  horizontal  line  from  the 


piobibility  value  and  reading  the  corresponding  r  value,  r  .  The  standard 

J  >  GSt 

error,  s.d.  Is  added  and  subtracted  from  the  value  eat>  The  points 


a-(ri) 


r.  .  +■  a 
),  est 

arc  lotnetl  to  form  another  curve. 


are  joined  to  form  one  curve;  the  points  r 


J,  eBt 


s.d 


•(h) 


To  complete  the  curves,  it  is  necessary  to  find  the  standard  error  of 
some  of  the  extreme  values.  The  asymptotic  distribution  of  the  extremes  is 
not  normal  and  is,  In  fact,  very  complicated.  In  Table  EX  are  given  some  values 
of  the  reduced  standard  error  for  the  largest  reduced  order  statistic.  By  the 
symmetry  of  the  normal  distribution,  this  is  also  the  standard  error  for  the 
smallest  reduced  order  statistic. 


The  values  in  Table  IX  are  approximated  In  the  TEN  COB  program  by 

'.m*  lor  mala, 


s.d.  -  B*d.  (vt>  -  0.  71  -  0.061  fne  (n)  , 

which  is  fairly  accurate  for  samples  of  20  to  500  points.  The  standard  error  of 
the  reduced  largest  (or  smallest)  value  ia  multiplied  by  the  estimate  1/a  to 
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TART.F  fX  STANDARD  ERRORS  FOR  REDUCED  LARGEST 
AND  SMALLEST  VALUES 


Sample  Size 

t-d*(Vn)  "  8,d>  ('V 

. 

20 

0,52 

26 

0. 51 

30 

0.R0 

40 

0.48 

5C 

0.46 

76 

0.45 

loo 

0.43 

2  nr- 

0.10 

,-r„ 

0.  36 

obtain  s.d.  [or  s.d.  (r,  t].  This  value  1b  then  added  to  and  subtracted 

from  r  .  or/V,  A  to  give  the  ]K>ints  r  . +  s.d.  (r  \  and  r 

n,  eat  y  1,  estj  *  n,  est  ^  n)  n,  est 

+  s.d.  ^r^  orjrj  egt  +  s.d.  (ri)andrj  ^-a.d.  (r^J.  These  points  are 

added  to  the  proper  curves  to  extend  the  oontrol  curves.  There  is  a  probability 

of  0  63  for  each  Jth  observation  to  lie  within  the  band  formed  by  the  two  control 
curves.  Figure  9  showB  the  68-percent  oontrol  band  for  Example  I. 

Multiplication  of  s.d.  by  0.6745,  1.960,  2.576,  2.807,  and  3.290 

leads  to  bands  corresponding  to  the  probabilities  0.50,  0.95,  0.99,  0.995,  and 
0. 999.  In  Figure  10  the  95-percent  control  band  has  been  added.  This  Is  the 
standard  graph  produced  by  the  TEN  COB  program. 


ri.  Testing  for  Normality  by  Control  Bands 

The  method  of  control  bands  used  by  the  TEN  COB  program  gives 
a  graphical  criterion  for  the  goodness  of  fit  between  the  theoretic  normal 
distribution  and  the  observations. 

If  almost  all  of  the  observations  fall  within  the  95-percent  band,  the 
underlying  distribution  can  be  assumed  to  be  normal  for  most  purposes  and 
statistical  tests  and  confidence  intervals  that  depend  upon  an  underlying  normal 
distribution  (such  as  the  Student's  t,  the  Chi-square,  and  the  F  tests)  can  be 
applied.  If  almost  all  of  the  observations  fall  within  the  68-percent  band,  more 
confidence  can  be  placed  on  the  population's  being  normal.  The  term  "almost  all" 
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is  necessarily  vague  since  the  degree  to  which  the  distribution  must  match  a 
normal  one  varies  with  each  set  of  observations. 

in  Example  i,  bv  percent  ot  tne  residuals  fell  outside  the  de-percent 
control  curve  and  l«  percent  fell  outside  the  95-percent  control  band.  Thus, 
the  data  are  probably  not  normally  distributed. 

When  the  observations  are  shown  to  he  not  normally  distributed,  one 
of  two  methods  can  be  employed.  The  data  can  be  tested  against  other  type*  of 
distributions,  such  as  the  negative  exponential,  the  log  normal,  etc.  There  are 
disadvantages  to  this  approach.  For  one,  the  list  of  distributions  to  be  tried  is 
long,  Further,  even  if  a  distribution  it'  found  that  will  approximate  that  of  the 
observation,  it  woult  not  have  aa  many  well-known  associated  tests  and 
procedures  as  the  nonnal  distribution. 

The  other  appr  ach  is  to  find  a  transformation  of  the  observations  that 
will  result  in  normally  distributed  transformed  observations.  A  list  of 
suggested  transformations  can  be  found  in  Snodecor  and  Cochran  [5] . 


9.  REVISED  MODEL  FOR  THE  MISSILE  DATA 

Since  the  residuals  for  the  radial  miss  distance  of  Example  I  were 
not  normally,  distributed,  it  was  decided  to  try  transforming  the  miss  distances 
to  obtain  normality  of  the  residuals.  The  transformation  i  «  in  y  was  made  and 

the  data  were  used  in  the  GELSF  program.  The  resulting  calculation  are  shown 
in  Tables  X  through  Xin.  The  prediction  equation  was: 

in  y  =  2.758  -  0.00947  -  0.  0363  x2  +  0.  000632  xs  +  0.0165  x4 

-  0.  000726  xs  +  0.00301  X*  +  0.  0910  x7  +  0. 250  x,  -  0. 167  x, 

+  0.  134  x]0  . 

The  residuals  were  plotted  by  the  TEN  COB  program,  as  shown  In 
Figure  11.  These  residuals  appear  to  be  normally  distributed.  Thus,  this 
was  the  model  upon  which  the  conclusions  were  drawn. 


10.  INTERPRETING  THE  RESULTS 

The  probability  distribution  of  the  residuals  when  the  logarithms 
of  the  radial  miss  distances  are  used  in  Example  I  Is  approximately  normal. 
Thus,  several  methods  of  testing  hypothesis  and  setting  confidence  regions 
are  applicable.  A  few  of  these  are  discussed  below. 
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a.  Testing  =  b*  Using  a  Student's  t  Random  Table 


av. - 1 - - .4 — I..  aL. 

*  NJ*-i4JWA  J/iWfJitMit  piUlVP  WUC  WUVCUIOUWC  1UOU1A  VI  UJC 

estimates  b  through  b  of  the  coefficients  of  the  linear  statistical  model.  Hie 
th  ^ 

1  diagonal  element  of  the  covariance  matrix,  c^,  estimates  the  variance  of  b^. 

b  -  b 
1  1 

The  ratio  — ==•  has  a  Student's  t  distribution  with  n-k-1  degrees  of  freedom, 
vii 

Suppose  that  a  predetermined  estimate  b*,  of  bj,  the  true  value,  Is 
available  and  the  following  hypotheses  are  postulated: 

H,:  bj  -  b;‘;  H,  ^  *  b*  . 

This  is  a  two-sided  test  since  there  is  no  advantage  if  <  b*.  As  before,  the 

level  of  the  test  is  a.  The  critical  t  value  la  found  for  the  level  of  the  test,  the 
number  of  degrees  of  freedom  and  the  fact  that  it  Is  a  two-sided  test, 

t  n-k-1  Since  the  t  distribution  Is  symmetric,  the  probability  that  a  t 
random  variable  is  greater  than  t^*“,  n-k-lj  or  lesa  than  -t^£,  n-k-1^  la  a, 

Prob  j^ft  |  >t^J,  n-k-l^j  «o 

where  it  |  is  the  absolute  value  of  t.  This  defines  the  orltloal  region  of  the  teat. 
bt  -  bj 

If  K0  is  true,  then  wj —  Is  a  t  random  variable  and 
VCii 

s  -bf 

Prob  -~ss=-  >  t 

W 

m 

Thus,  the  decision  is 


(f ,  »-k-i) 


»  a  . 


Reject  H0  if 


V? 


>  t 


FT 

Do  not  reject  H0  otherwise. 


(f ,  n-k-l)  . 
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11  it  is  desired  *<>  whether  the  variable  has  a  "significant  eficct "  ujxm  y, 
then  set  "rr  -  u  ami  no  norm  the  above  ie-u. 


b.  Testing  the  Coefficients  for  Missile  Modifications 

In  the  revised  model  for  the  missile,  using  z  -  in  (radial  miss 
distance),  the  coefficient  b7  represents  the  difference  in  z  caused  by  a  difference 
In  Mod  1  and  Mod  2  missile.  That  is,  b:  =  z  .  -  z  ,  .  From  Table  X» 

the  estimate  of  this  coefficient  is  Mod  2  M  1 


by 


0.0010 


ZMod  2  "  ZMod  1  ‘ 


Since  »  =  fn  ■/;,  a  positive  increase  In  z  corresponds  to  a  positive 
increase  in  y,  Hium,  it  would  appear  that  the  radial  miss  distance  would  be 
larger  for  Mod  2  missile  than  for  Mod  1  missiles.  To  see  whether  this 
difference  is  statistically  significant,  a  t  test  is  performed,  with  b*  =  0. 

It  was  decided  lo  set  o-  0. 10.  The  number  of  degrees  of  freedom  is, 
from  Table  X,  89.  The  hypotheses  are:. 

Hq!  b7  =  0,  Hj!  b7  $  0  ■ 

The  critical  t  value  ie 

t  (0.05,  89)  »  1.84  . 

The  estimate  of  the  variance  of  fi7  is,  from  Table  XII, 

Cyy  =  0,  1804  . 

The  tost  statistic  is 

b7  ~  0  0.0910 

r  *  »  TA  n  —  0.  2 1  ■ 

VCyy  ^0.  1804 

The  test  statistic  is  less  than  the  critical  value  of  1. 64,  Therefore,  it  cannot 
be  concluded  that  there  is  a  significant  difference  in  the  radial  miss  distance  for 
Mod  2  and  Mod  1  missiles. 


A  similar  test  is  made  for  the  difference  in  z  of  Mod  3  and  Mod  2 
missiles.  Again,  s«  0.10,  t  (0.05,  §9)  =  1.64.  The  hypotheses  are  HP:  b8  =  0. 
Hp  be  *  0.  From  Tables  X  and  XII,  b3  =  0.  250,  cgs  -  0. 0740.  The  test 
statistic  Is  thus 


This  Is  less  than  the  critical  value.  It  cannot  be  concluded  that  there  is  a 
significant  difference  In  the  effect  of  Mod  3  and  Mod  2  on  radial  miss  distance. 

There  la  one  more  difference  to  be  examined;  that  between  Mod  3  and 
Mod  1.  To  estimate  this  difference,  the  following  sum  la  used: 

h  *  * 

7  =  ZMod  2  “  ZMod  1 

^  a  ZMod  3  "  *Mod  2 
b7  +  68  =  2Mod  3  -  *m0(j  1 


In  this  case 

6t  +  a  0. 0910  +  0. 250  ■  0. 341  . 

Since  b7  and  b8  are  normally  distributed,  then  b7  +  b8  is  also  normally  distributed 
with  mean  b7  +  b3  and  variance 

var  (fi7  +  6,)  »  var  (67)  +  var  <db)  +  2  cov  (fi7,  fi8)  . 

This  variance  is  estimated  by 

/  c77  +  css  +  2c78  . 

From  Table  XII,  c7B  =■  0.0770.  Thus 

c77  +  cn  +  2c78  ■  0. 1804  +  0. 0740  -I-  0. 0770  »  0. 3314  . 

To  test  the  hypotheses 

H#:  b7  >  b8  »  0;  Hj:  b7  +  b8  #  0  , 


for  a  =  o.  10,  the  critical  value  is  t  (0.05,  89)  ■  1.64.  The  teat  statistic  is 


6,  +  6,  -  0 


/c7T  +  ce,  +  2cT8 


0.341 

to m 


0.59  . 


Again,  there  is  Insufficient  evidence  of  a  difference  in  miss  distance  between 
Mod  3  and  Mod  1. 
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c.  Using  the  F  Test  in  Analysis  nf  Variance 


Suppose  that  a  model,  called  Model  I  or  the  complete  model,  has 
been  fiti< to  a  set  of  data  and  it  la  desired  to  reduce  the  model  by  dropping  all 
terms  which  do  not  test  as  significant.  Assume  that  the  level  of  each  test  is 
0. 05  and  that  the  tesls  are  independent.  Ii  onlv  one  term  is  tested  without 
positive  results  by  at  test  and  dropped  from  the  model,  then  the  probability  of 
falsely  rejecting  is  0.05.  If  two  terms  are  tested  separately  with  t  tests  and 
dropped,  the  probability  of  falsely  rejecting  ut  least  one  coefficient  becomes 
1  -  (0. 95 ) 2  a  0. 0975.  If  four  terms  are  tested  and  dropped,  the  probability  of 
falsely  rejecting  at  least  one  coefficient  is  1  -  (0. 95)4  »  0. 1855.  If  this  1b 
extended  to  eight  terms,  it  become  a  1  -  (0. 95 > ^  =  0.3386.  Thus,  the  t  test 
ran  bo  safely  used  only  when  one  term  alor  -  is  to  ba  dropped.  If  more  than  one 
term  Is  if  he  testei  ,  the  method  of  anal  sie  of  variance  should  be  used. 

bet  Me  lei  *  (the  complete  model)  he  the  following: 

Model  I:  y  =  b0  +  b ,  x t  +  bj  x2  +  ...  +  b^  +  e  . 


Model  I  has  been  fitted  to  the  data  and  the  sum  of  squares  of  error,  SSE  t,  and 
the  estimate  of  the  varisnoe,  Sj2,  have  been  calculated. 

Now  suppose  that  it  Is  desired  to  test  the  significance  of  k-g  of  those 
terms.  For  convenience,  assume  that  the  last  k-g  terms  are  to  be  tested. 

The  null  hypothesis  la  then 


If  is  true,  then  the  system  can  be  described  by  a  shorter  or  deleted  model, 
Model  II: 


Model  n: 


y  »  bn  +  bj  x(  +  . . . 


+  b  x  +  e  . 
S  K 


Model  II  is  then  fitted  to  the  data  and  the  sum  of  squares  of  error,  SSE2, 
and  the  estimate  of  the  variance,  s<>},  are  calculated. 


Even  if  H0  Is  true,  the  estimates  of  h^  +  ^  through  will  not  be  zero 

because  of  the  finite  random  sample  of  observation  noise.  Thus,  fitting  Model  I 
Instead  of  Model  II  will  reduce  the  sum  of  squares  for  error,  SSE  ^  as  SSE2 . 

In  fact.  SSEj  can  be  partitioned  into  two  positive  quantities 


SSEg  a  SSE  j  +  (SSE j  —  SSE ,)  , 
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SSE  SSE  —  SSE 

It  can  be  shown  that  if  H0  is  true,  — —  and  ■  *  ■  — ■  1  provide  unbiased 

0  n-k-l  k-g 

estimates  of  o2  and  that  ~jjli  and  001,2  are  independent  Chi-square 

random  variables  with  n-k-l  and  k-g  degrees  of  freedom,  respectively.  Thus, 
the  ratio  of  these  two  independent  Chi-square  random  variables  divided  by  the 
respective  degrees  of  freedom  is  an  F  random  variable,  If  H0  is  true. 


SSE,  -  SSE, 
(k-S) 


7 


SSE 


L 


P~  (n-k-l) 


SSE t  -  SSE, 


To  test  H0,  a  01  e- sided  test  on  this  F  ratio  is  used.  The  level  of  the  teBt 
nt  must  again  be  specified  The  critical  value  c f  F  depends  upon  a  and  upon  the 
degrees  of  freedom,  k-g  and  n-k-lj  =  F  (a,  k-g,  n-k-l).  This  is  found 

in  statistical  tables.  vr 

The  decision  It  then  based  upon  a  comparison  of  F  with  the  critical 

value: 


If  F>  F  (a,  k-g,  n-k-l),  reject  H„. 

If  F  25  F  (a,  k-g,  n-k-l) ,  do  not  reject  H0  . 

If  H0  is  rejected,  the  model  cannot  be  shortened  by  dropping  the  entire  group  of 
k-g  terms.  Possibly,  a  subgroup  of  these  k-g  terms  can  safely  be  dropped,  but 
other  modelq  must  be  postulated  and  tested  to  decide  whioh  can  be  dropped. 


d.  Testing  the  Effect  of  Missile  Modification  Upon  Radial  Miss 
Distance  Using  the  F  Statistic 

A  reduced  model,  without  the  terms  for  missile  modification, 

was  set  up  as: 

z  -  bn  +  h,  x,+  b}  J!j.+  b3  Xj  ♦  b*  x4  +  bs  x5  +  b,  x,  +  b,  x, 


+  b10  x,o  +  c  . 

The  estimated  coefficients  for  this  model  are  shown  in  Table  XIV. 

The  null  hypothesis  for  this  test  is  H0:  b7  ■  b,  =  0.  It  was  decided  to 
set  a  *  0. 10.  The  critical  F  value  la 
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TABLE  XIV.  THE  ESTIMATED  COEFFICIENTS  FOR  THE  REDUCED  MODEL  USING 
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A 


J 


F  (0.10,  2,  89)  =  2.77 


PfOTTl  Tnbl6  X  for  tho  CO!^.nl6tA  mnHol 


SSE1  «■  35. 1213 
a  i2  =  0.  3946  . 

From  Table  XIV  for  the  reduced  model , 


SSE2  =  35.5674  . 


The  test  statistic  is 

■:SE;  -  SSE,  35.5674  -  35. 1213  _ 

"(k-g)  8, 2  =  ‘  (2  X  0.3946)  "  °,S  • 

This  value  is  less  than  the  critical  value.  It  cannot  be  concluded  that 
missile  modification  affects  the  radial  miss  distance. 


i 


11.  CONCLUSIONS 

'"1 

Often  observations  or  results  of  experiments  can  logically  be 
represented  by  a  linear  statistical  model  relating  the  observation  y  to  various 
known  quantitative  and  qualitative  variables  x„  .. . ,  x^,  random  noise  e, 

and  fixed  but  unknown  coefficients  bj,  b,  . , .  ,  b,: 


y  *  b0  +  b,  x,  +  b2  x2  +  ... 


+  bkVe- 


\ 

The  least  squares  estimates  of  the  coefficients  b0,  b1(  . . .  ,  b^  have  been 
developed  theoretically  and  are  calculated  by  the  GELSF  program. 


If  the  noise  e  is  Gaussian  (normal)  with  zero  mean  and  covariance 
matrix  0*1,  where  I  is  the  identity  matrix,  then  certain  quantities  have  well 
known  distributions.  A  graphical  procedure  for  making  tests  for  normality  of 
the  noise  la  described. 


Crucial  In  the  entire  discussion  is  the  model.  The  model  must  be  of 
the  correct  form  or  the  theory  collapses.  Statistical  tests,  in  particular  the 
analyslo  of  variance,  can  suggest  which  terms  should  be  retained  in  the  model, 
but  they  cannot  prescribe  which  new  terms  should  be  added.  The  model  must  be 
constructed  from  physical  considerations  and  sound  Judgment. 


! 
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Sound  judgment  should  also  be  employed  in  the  statistical  evaluation  of 
the  data.  In  no  case  should  etatlstical  techniques  described  here  or  elsewhere 
be  applied  in  a  purely  mechanical  manner  or  divorced  from  the  other  aspects  of 
the  system  under  consideration. 
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1.  introduction.  Frequently,  experiments  are  conducted  in  such  away 
that  they  may  be  considered  to  consist  of  n  tri  lls,  where  in  eac.h  trial, 
the  time  required  to  achieve  some  objective  (  "success"  )  is  observed. 

If  an  upper  limit,  '  .  is  placed  upon  the  possible  duration  of  each  trial, 
then  the  outcome  on  'iadt  trial  is  either  the  t  ine  until  success,  or  Tq  . 
whichever  is  p mallei .  Such  observations  are  said  to  be  censored  at  Tq  . 

For  example,  it  might  be  desired  to  determine  whether  a  certain  type  of 
combat  aid,  such  as  a  target  detection  device,  is  effective,  or  whether 
one  type  of  device  is  better  than  another.  In  order  to  make  such  inferences 
about  a  single  device,  often  an  experiment  of  the  following  design  Is  conducted: 
n  players  are  selected,  and  each  player  uses  the  device  in  an  attempt  to  / 
detect  a  target.  If  a  given  player  has  not  suceeded  within  4  minutes  (say). 
Biatrial  terminates  and  the  next  player  begins.  The  observed  data  then 
consists  of  the  times  of  detection  for  those  trials  terminating  before 
T  =  4  (together  with  the  number  of  trials  terminating  at  Tq  =  4).  If  it  iB 
desired  to  coinpare  two  devices ,  then  samples  on  each  device  may  be  taken 
as  described  above. 

If  the  distributions  of  (uncensored)  time  until  success  are  identical 
exponentials,  this  situation  falls  under  the  body  of  results  generally  known 
as  "life  testing."  In  what  follows,  we  shall  discuss  statistical  procedures 
for  making  inferences  about  the  mean  rate  X  of  success  (the  reciprocal 
of  mean  time  to  success),  bused  upon  such  censored  time  dependent 
observations.  These  inference  techniques  Include  point  estimators, 
confidence  intervals  and  tests  of  hypotheses  for  both  the  single  population 
esse,  and  that  of  comparing  two  populations.  Since  there  is  a  strong 
parallel  between  making  a  confidence  interval  for  a  parameter  and  testa 
of  hypotheses  concerning  that  parameter,  we  shall  discuss  only  one  or 


This  ax  tide  has  been  reproduced  photographically  from  the  authors1 
manuscript . 


the  other  of  these  in  enrh  nf  the  approaches  considered  l>elow.  While  it  is 
assumed  that  the  populations  involved  are  exponentially  distributed, 
attention  is  given  to  the  robustness  of  the  procedures  under  departures 
from  exponential  distributions. 

In  Section  2,  we  discuss  procedures  for  making  inferences  about 
one  population  (one-sample  inferences),  including  a  review  of  several 
approaches  in  the  literature  and  an  easily  applied  approximate  procedure 
that  wc  have  been  investigating.  A  similar  treatment  for  two-sample 
inferences  is  given  in  section  3. 


2.  One-Sample  Pro,  ^djieg.  Suppose  that  Z.,  ...,Z  is  a  random 

sample  of  size  n  from  an  exponential  population  with  mean  rate  X  . 

Let  -  min  {  Z.  ,  Tq  }  ;  i  =  1,  2 . n.  Based  upon  observations 

x,,,..,x  on  the  X*  s.  we  wish  to  make  inferences  concerning  the 
1  n 

parameter  X  .  The  (point)  estimation  of  X  has  been  discussed  by 
several  writers  (see.  for  example.  Bartholomew  [  1]  ) .  Theyhnve  shown 
that  the  maximum  likelihood  estimate  of  X  is 


i  .  k/Si{alx1Ml-»,)T0  }  . 


where 


li  *  { 


1  if  x,  <  T  .  _  ,  „ 

0  othekvise0  ’  1  "  2 .  ’ 


and 


n 

Vi  • 

1-1 


While  1/X  is  asymptotically  normal  with  mean  1/X  and  variance 
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LnX S(1  -  c'X  r°  J  ,  the  convergence  is  slow  [  l]  and  ,  for  small 
samples,  X  is  seriously  biased  [0]  .  The  exact  distribution  of  1/X 
la  given  by  Bartholomew  [  1]  in  a  form  useful  with  small  samples, 
along  with  some  approximations  that  arc  useful  with  moderate  sized 
samples. 

Several  procedures  for  obtaining  confidence  intervals  for,  X 
and  1/X  ,  based  upon  approximate  distributions  for  certain  functions 
of  1A  ,  have  been  suggested.  Bartholomew  [  1]  discusses  two 
procedures  based  up<  n  normal  distributions,  and  a  technique  given  in 
NAVORD  O.  D.  .29304  i  <1 3 uses  an  approach  based  upon  a  Poisson  distribution. 
The  statistical  properties  of  confidence  intervals  obtained  by  these  approxima¬ 
tions  are  apparently  not  fully  known  at  the  present  time. 

We  have  investigated  a  method  for  obtaining  confidence  intervals 
for  X  .  based  upon  a  general  approach  given  by  Halperin  [4]  ,  described 
as  follows:  Let 

p  -  1  -c"XT°  (1) 

denote  the  probability  that  the  result  In  a  given  trial  Is  not  censored 
(1.  e. ,  p  ■»  P  [  Xj  <  TQ  ]  )  .  Then  each  experimental  trial  may  be 
viewed  as  a  Bernoulli  trial,  where  "success"  is  associated  with 
non-censoring  and  occurs  on  each  trial  with  probability  p  .  Based 
upon  the  observed  number  k  of  success  in  the  n  experimental  trials,  ’ 
a  100  ( 1  -a)  percent  upper  confidence  bound,  say  ,  for  p  can  be 
constructed  using  well-known  methods.  Using  equation  (1),  this  bound 
can  be  "inverted"  to  obtain  a  100  (1-a)  percent  upper  confidence  bound 
for  X  as 


JL.  .  '"'"-’V 

U  rr 


(2) 
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Intervals  and  bounds  of  this  type  are  very  easy  to  compute,  and  appear  to 
perform  nearly  as  well  as  those  based  upon  approximating  distributions. 

The  results  of  a  Monte  Carlo  study  on  the  performances  of  the  two  procedures 
based  upon  the  Poisson  distribution  [6]  and  the  binomial  distribution  [4] 
are  presented  In  Tables  1,  2,  and  3  below.  1000  samples  of  size  n 
(n  =  20.  30,  40,  50)  were  generated  from  exponential  distributions  with 
mean  rate  X  (X  -  .1.  .2.  .5,  1,  2,  5.  10).  For  different  truncation  times 
Tq  the  upper  confidence  limits  X^  were  calculated  using  both  procedures. 
Table  1  gives  the  average  value  of  the  upper  confidence  limit  Xy  .  ,For 
each  choice  of  Xand  T0  the  first  row  contains  X^  for  the  O.  D.  29304  [6] 
procedure  (based  on  the  poisson  distribution);  the  numbers  in  the  second 
row  are  those  for  the  binomial  procedure.  Table  2  contains  the  sample 
variance  of  the  upper  bounds  X^  ■  Jn  Table  3,  the  empirical  ooverage 
probability  i.e,  ,  the  proportion  of  the  Xy  which  actually  exceed  the  true 
parameter  value  X  is  given,  The  relative  sensitivity  of  these  procedures 
to  departures  from  the  exponential  distribution  are  apparently  not  known 
at  present. 

Finally,  we  mention  another  approach  to  finding  confidence  intervals 
for  X.  which  seems  to  have  received  less  attention  In  the  literature  than  it 
deserves.  Imagine  that  the  time  censored  trials  are  conducted  sequentially 
in  time,  and  that  we  disregard  the  times  between  when  each  trial  terminates 
and  the  next  begins  (see  Figure  1).  Then  the  times  between  successive 
successes  is  exponential  with  parameter  X  .  so  that  the  "success  arrival 
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Sample  Size  n 

20 

..  3° 

40 

50 

Confluence  Coefficient  1 

< X 

J 

A 

To 

.95 

.  99 

.  95 

.  99 

.  95 

.  99 

.95 

.  99 

.  1 

8. 

■ 

.  199 

■ 

.  144 

.  164 

.  138 

.  156 

1 

.209 

.  149 

.  108 

.  142 

.  159 

.2 

2. 

.379 

.477 

.316 

.  375 

.299 

.352 

.397 

.478 

.  352 

.408 

.324 

.370 

.  305 

.352 

.  5 

n 

.  919 

1, 12 

- - 

.  804 

.  975 

.  752 

.  890 

.723 

.840 

1.13 

.  836 

.  985 

.772 

.  904 

.738 

.  846 

B 

■ 

1.08 

>.05 

1.53 

1.  82 

1.44 

[■- -  - 

1.  G8 

1.  40 

1.  60 

HI 

*  1.78 

2.  13 

1.  00 

1.  80 

1.49 

1.71 

1.43 

■KHI 

2. 

.3 

3.50 

4.28 

3.  74 

2.97 

3.  47 

2.  84 

3.26 

3.72 

4.39 

3.23 

3.82 

3.  52 

2.  92 

3.30 

5. 

.05 

1 

14.  5 

9.  46 

12.0 

8.53 

10.  7 

8. 14 

9.95 

■RnvHI 

14.3 

11.9 

8.76 

8.33 

9.88 

10. 

.06 

mrm 

21.6 

15.6 

18.8 

14.6 

— 

17.2 

14.1 

16.3 

2,2.  2 

16.2 

19.2 

15.1 

17,  3 

14.  r» 

16.5 

TABLE  1.  Average  upper  100  (  1  -  a  )  percent  confidence  bounds 
Ay  for  the  0.  D.  and  the  binomial  procedures. 
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Sample  Size  n 

30 

40 

50  | 

Confidence  coemcient  1  - 

Q 

X 

To 

,  95 

.  99 

.  95 

.99 

.95 

.  99 

.95 

.99 

1 

8. 

.002 

.002 

.001 

ggmi 

.  003 

.004 

.  001 

HB 

bhehiBH 

.2 

2. 

.  012 

.015 

.008 

* 

004 

.014 

.017 

.  008 

ISilffTftB 

_ 

•iin 

■HnTTBRif 

.5 

1. 

.067 

.076 

.036 

.  045 

.028 

.033 

.022 

.  080 

.094 

.  042 

■mim 

.030 

MM 

1. 

.  7 

.  194 

.267 

.  124 

K3 

.086 

.096 

.063 

.074 

,  265 

.  356 

.  146 

■Kl£9 

.119 

«i 

2. 

3 

.  874 

1,  Ob 

,  44« 

HI 

mm 

.485 

. 

264 

.353 

1.  11 

1.46 

.  578 

.  769 

mr% 

.567 

. 

.386 

5. 

.05 

12.  7 

16.8 

7.  30 

9.  34 

1 

6.68 

3. 

88 

l  ' 

4.  74 

14.  0 

18,  5 

7,  80 

10.  2 

1 

6.96 

4. 

OSH 

4^6 _ 

10, 

.  00 

19.  8 

29.4 

m  7 

16.  8 

9.  15 

11.0 

6.78 

mm 

25.  V 

40,  1 

lit.  2 

20.0 

10.  6 

12.4 

7.60 

ISnBH 

TABLE  2.  Sample  Variances  of 
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TABLE  3.  Proportion  of  the  that  exceed  the  true 
parameter  value  A  . 


Outcome*  of  trials  j 

U»  U*  relative  : 

fr*qu**cy  model  ) 


"  typical  experimental 
triaia  (maximum  length: 
minute*  p»ch) 


Arrival*  of  "auceesati"! 
in  uw  mean  umt  model  \ 


•  • 


FIGURE  1. 

process"  is  a  polsson  process  with  mean  rate  X  per  unit  time.  Suppose 
that  k>  0  successes  are  observed  in  n  experimental  trials.  Then  the 
conditional  distribution  of  the  waiting  time  to  kth  arrival,  ,  given 
k,  is  approximately  gamma  distributed.  Thus  iiXW^  iBuhl-square  distributed  with 
2k  degrees  of  freedom.  Using  the  tubulated  \a  distribution,  and  given 
k  >  0  ,  one  can  easily  find  an  interval  <CL  ,  Cy)  which  will  contain 
2XWk  with  probability  1  -a  .  Algebraic  manipulation  upon  this  interval 
yields 


as  a  100  (1-a)  percent  conditional  confidence  interval  for  X  . 

The  sensitivity  of  this  approach  to  departures  from  exponential  distributions 
depends  upon  T0  ;  with  sufficiently  small  T0  ,  the  interarrival  times 
should  bo  nearly  exponential  even  with  non- exponentially  distributed 
times  to  success  within  the  individual  trials. 
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Two-Sample  Procedures,  la  this  section,  techniques  for  comparing 
two  exponential  populations  based  on  censored  observations,  are  discussed. 
Suppose  independent  samples  of  sizes  and  n 2  are  drawn  from  two 
exponential  populations  with  means  1/Xj,  and  1/Xg  respectively,  It 
will  be  assumed  that  both  sets  of  sample  observations  are  censored  at 
the  same  time  point  T0  ;  that  is,  an  upper  limit  T0 ,  is  placed  -upon 
the  possible  duratior  of  each  trial.  Let  ki  and  ks  denote  the  number 
of  uncensored  observations  in  the  first  and  second  sample  respectively 
and  let  p-  kx4ka.  Methods  for  testing  the  hypothesis  Hq  :  XA  -X3 
and  obtaining  confiden-  e  intervals  for  p  Xi/As  and  \ii  -  X1-Xa  are 
discussed  below. 

a.  F  test  for  HQ  :  X!  -X8  :  Let  k  denote  the  total  elapsed  time 
time  until  k:  uncensored  observations  ore  obtained  from  the  first  sample. 
Then,  as  mentioned  earlier,  2X,  W.  ,  has  a  chi-square  distribution  with 

1  1,  Kj 

2  k1  degrees  of  freedom.  If  W2  k  is  similarly  defined  for  the  second 
sample,  so  that  2A„W0  .  is  distributed  as  a  chi-square  with  2kr)  degrees 

A  A  i  K  j  A 

of  freedom,  then,  if  Ho  Is  true,  the  ratio 

*  t 

W  =*  2wl,kv/2kj.  (4) 

2W2,k|/2ki. 

has  an  F  distribution  with  (2k;,2ka)  degrees  of  freedom.  The  hypothesis 
H0  will  be  rejected  in  favor  of  Hi  :  Xj<Xg  .  at  significance  level  at  . 
if  the  observed  value  W  exceeds  the  lOOath  percentile  of  the  F  distribution 
with  (2kj,  2kg  )  degrees  of  freedom.  A  confidence  interval  for  p  *  Aj/Xg 
can  be  obtained,  based  on  the  F  distribution  of  kgAjWj  k  /kjA 2  k#  ■ 
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b.  Cox's  F' test  :  Let  n  =  n,  +  n„  and  let  the  scores  t 

“  r  i  n 

(  r  -  1.  2,  ....  n  )  denote  the  expected  values  of  the  order  statistics 
of  a  random  sample  of  size  n  from  an  exponential  distribution  with 
mean  equal  to  1.  It  can  be  shown  that 

r1"1 
r.n  L  s=  0 

Combine  the  p  -  uncensored  observations,  defined  in  the 

beginning  of  the  section  and  rank  them.  Replace  the  observation  with 

rank  r  with  the  corresponding  score  t  (  r  - 1, 2, .  .  .  ,  n)  .  if  two 

or  more  of  the  censored  observations  are  equal,  replace  each  otte  with 

the  average  of  the  corresponding  scores  t  .  Let  f ,  denote  the 

rn  l 

average  of  the  scores  assigned  to  the  observations  fron  the  first  sample 
and  tg  tho  average  of  the  scores  assigned  to  the  observations  from  the 
second  sample.  Cox  [2]  has  shown  Mutt  the  ratio 


J—  («•  =  !.  2 . n  ) 

n  -  s 


["  k,  t.  +  (n,  -k, )  t_  .  t  _  l/k- 
VV  '■[**""""■  ~"—'1  . . 1,n  jf  ,  1-  ■  (5) 

L  k2*2  "  <n2  "  k2)  V  +  l.n  J/k2 

is  approximately  distributed  as  an  F  with  (2kj  ,  2kg)  degrees  of 
freedom,  when  HQ  i  'A2  1®  true.  The  rejection  region  for  the 
hypothesis  H0  :  A2  can  be  determined  from  the  F  tables. 

A  confidence  Interval  for  p  -  A^A2  can  be  obtained  using  this 
approximate  distribution  of  W*  ,  as  follows:  Multiply  each  of  the  uncensored 
observations  from  the  second  sample  by  a  fixed  number  Pfe  and  apply 
Cox's  procedure.  This  will  lead  to  a  test  of  the  hypothesis  H'  :  A.  -  p  A_  . 

O  X  0  4 
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For  a  given  significance  level  a  ,  the  set  oi  all  values  of  p3  that 
will  lead  to  rejection  of  H'  will  from  a  confidence  interval  for  p  with 
confidence  coefficient  1  -a . 

Recently,  Cehan  and  Thomas  [3]  have  reported  a  Monte 
Carlo  study  comparing  the  powers  of  the  F  and  F'  tests  lor  small 
sample  sizes.  It  was  found  that,  when  the  assumption  that  the  two 
samples  arc  from  exponential  distributions  is  valid,  these  tests  have 
comparable  operating  characteristics,  (see  Figure  2  below)  However. 
If  the  samples  are  from  Wclbull  distributions,  the  K  test  is  nut  robust 
and  the  F'  test  is  superior.  It  should  be  noted  that  the  I'1  te'st  requires 
that  both  sets  of  sample  observations  are  censored  at  the  same  time- 
point  Tq  ;  the  F  e’st  is  not  constrained  with  this  requirement. 


U|S>n«>itt  i  1 1 ;  ton  statu 
iU-tf.  t  |>m  r.iie 


4  . 


UcrM^in*  .lettcUon  rat** 


re  r?  Th  ,  *, 


De-rnilng  del 


FIGURE  2.  Operating  Characteristic  Curves  for  the  F  and  I"  Tests 
With  Censored  Observations  for  Expone.v.ia.  and  Ce.-ii.r.  V,«. .  ..  . 

Distribution,  (n.  -  nn  -  20,  a  -  ,0S  and  A,,  -1.0) 


c.  Confidence  interval  ior  e  A^  -  Ag  :  Suppose  W ^  und  Wg 
are  independent  random  variables  with  gumma  distributions  with  parameters 
\l  and  A„.  Center  and  Buehler  .1]  obtained  the  conditional  distribution 
of  H  (u/v  ;  v  )  of  U  Wj  given  V  "W,  +  W,,.  This  conditional 
distribution  function  involves  only  the  parameter  =  A^  -  A9-  A  confidence 
interval  for  c  can  be  obtained  as  follows:  set  H  (  u/v  :  $  )  equal  to  a/2 
and  1  -a/2  respectively  and  solve  for  c  .  The  two  solutions  for  b  will  be  the  lower 
and  upper  confidence  limits  for  C  with  confidence  coefficient  1-a. 

The  Lenter-Buehlar  techni<|ue  can  be  used  to  derive  a  confidence 
interval  for  the  difterei.ee  Aj  -  A.,  for  two  exponential  distributions 
when  the  observations  are  censored.  As  was  pointed  out  earlier  in  this 
paper,  ii  the  censored  observations  front  an  exponential  distribution  are 
treated  as  having  been  obtained  sequentially,  then  2A  ,  where 
is  the  waiting  time  till  Kth  arrival,  given  k.  has  a  Chi-square  distribution 
with  2k  degrees  of  freedom.  Thus  has  a  gamma  distribution.  We 
can  now  form  two  gamma  distributed'  variables  from  the  two  sets  of 
observations  from  the  </xponential  distributions  und  apply  Lentor- Buehler  [3] 

•  technique  to  obtain  a  confidence  interval  for  A1  -  Ag  . 
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OPTIMIZING  A  FOUR-PART  ASSAY  PROCEDURE 


Walter  D.  Foster 

Biomathematics  Division,  Department  of  the  Army 
Fort  Detrick,  Maryland 


ABSTRACT.  Optimizing  the  procedure  of  a  four-part  assay  of  a 
bulk  material  is  considered  on  the  basis  of  the  variance  of  each  part 
versus'  its  cost.  In  d  replications  (defined  as  Days),  a  settling 
Chamber  is  used  c  times  to  deposit  organisms  on  p  Plates  per  chamber 
run;  the  plates  are  incubated  and  Read  r  times  per  plate.  A  nested 
analysis  of  variance  provided  estimates  of  the  components  for  the 
variance  function: 


V (Assay)  ■ 


The  cost  function  ia  the  sum  of  the  cost  rate  x  number  of  repeats  for 
each  part : 


Cost  ■>  (unit  Reading  cost)  rpcd  +  (unit  Plating  cost)  pcd 

+  (unit  Chamber  coat)  cd  +  (unit  Day  cost)  d. 

A  Lagranglan  multiplier  approach  yielded  minimum  variance  for  a  fixed 
total  cost: 


Min  [V  (Assay)  +  l(Coat)]. 

This  approach  is  contrasted  with  non-linear  programming  and  integer 
programming. 

INTRODUCTION .  A  problem  common  to  both  bulk  sampling  and  sample 
surveys  Is  to  obtain  as  much  information  about  a  population,  usually 
stratified  or  segmented  in  time  or  apace,  aa  possible  for  a  given 
budget.  For  example, , grain  production  in  the  midwest  or  even  from 
a  specific  farm  would  need  to  be  sampled  according  to  various  strata 
defined  by  bags,  bushels,  wagnn  loads,  etc.  To  maximize  the  Infor¬ 
mation,  i.e.,  minimise  the  variance  of  the  sample  mean  for  a  given 
coat  is  the  way  the  problem  ia  usually  stated. 

Duncen^,  among  others,  has  written  extensively  on  procedures  for 
bulk  sampling  and  has  a  pertinent  bibliography.  Similarly,  Cochran^, 
to  pick  another  outstanding  writer,  has  worked  extensively  in  the  area 
of  sample  surveys.  The  problem  addressed  here  is  to  optimize  an  assay 
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procedure  for  a  sporulating  organism  where  the  steps  stay  be  considered 
as  a  three-phase  procedure.  This  problem  is  answered  partly  by  bor¬ 
rowing  the  technique  from  bulk  end  sample  out vt/  auluiiona;  it  ia  also 
approached  from  the  point  of  view  of  integer  programming. 

A  "day"  ia  a  natural  block  of  work.  A  small  amount  of  the  spore 
aster ial  in  dry  state  is  encapsulated  and  dispersed  with  a  CO^  pistol 

into  a  small  chamber.  Several  agar  plates  are  set  in  the  bottom  of  the 
chamber  onto  which  the  dispersed  spores  fall.  After  a  suitable  incuba¬ 
tion  period  in  a  humidity  chamber,  the  plates  are  "read"  until  100 
organisms  are  counted  and  the  percent  viability  is  estimated.  Several 
such  readings  are  mada.  It  is  the  purpose  here  to  minimise  the  variance 
of  this  assay  procedure  subject  to  a  particular  assay  cost. 

I.  VARIANCE  FVNCTION .  An  experiment  was  designed  specifically  to 
provide  estimates  of  the  various  sources  of  variation.  In  each  of 
several  "days,"  repeated  chamber  runs  provided  an  opportunity  to  obaarve 
variation  from  one  chamber  run  to  another  within  the  same  day.  Six 
chamber  runs  per  day  were  achoduled,  three  agar  platea  per  chamber  run 
were  prepared,  and  aach  plata  was  read  aix  times.  The  experiment  con¬ 
tinued  over  18  days.  For  sake  of  simplicity  the  variation  due  to 
technicians  is  omitted  in  this  report. 

A  nested  analysis  of  variance  gives  both  the  structure  of  the 
analysis  and  the  results. 


ANALYSIS  OF  VARIANCE  ON  X  VIABILITY 


Source 

df 

MS 

EMS 

Days 

17 

315.013 

2a2  +  6o 
P 

Chambers  in  Days 

90 

26.990 

»!♦ 

2a2  +  6o 
P  1 

Plates  in  chambers 

216 

20.981 

»«  + 

Readings  in  pistes 

324 

15.012 

1 

4 

2  .  2 
4  • 15  >  4 

-  3; 

2  .  ,  2  . 

°c  “  1;  °D  " 

8 

Because  of  the  balanced  arrangement,  there  was  no  problem  in  estimating 
tha  components  in  thla  hierarchical  analysis  according  to  the  axpactad 
mean  square  shown  in  the  table  above.  The  estimates  of  thsas  components 
of  variance  are  also  shown  there.  In  this  completely  randomised  design 
with  sll  factors  considered  random,  the  variance  of  the  mean  percent 
viability  is  given  the  following  equation: 
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This  variance  function  permits  the  experimenter  to  obtain  any  degree 
of  precision  he  is  willing  to  pay  for  merely  by  assigning  appropriate 
values  to  the  number  of  readings  per  plate,  the  number  of  plates  per 
chamber,  the  number  of  chamber  runs  per  day,  and  the  number  of  days. 

One  criterion  for  this  degree  of  accuracy  is  the  cost  of  the  assay. 

A  second  criterion  concerns  the  Intended  use  of  the  material  and  to 
what  degree  of  accuracy  ha  must  know  its  viability. 

II.  COST  FUKCTION.  Cost  functions  are  notoriously  difficult  to 
describe  realistiCKlly ,  especially  when  the  assay  procedure  itself  Is 
conducted  on  a  sporcdlc  basis.  The  function  chosen  here  was  selected 
primarily  because  of  its  simplicity  and  was  felt  to  be  adequate  for 
the  problem  at  hand: 

S  ■  rpcd  R  +  pcd  P  +  cd  C  +  d  0,  where  R  ■  .20;  P  ■  .50;  C  -  .50; 
and  D  -  2.50  are  the  (fictitious)  cost  per  unit  for  Reading,  Plate, 
Chamber  and  Day,  and  r,  p,  c,  d  are  the  number  of  units  of  each. 

As  is  clear,  this  function  aleo  follows  a  nested  construction  and 
assigns  to  each  portion  of  the  assay  a  fixed  cost  so  that  total  coat 
is  obtained  merely  by  counting  the  number  of  component  parts .  Overhead 
costs  such  as  equipment,  utilities,  laboratory  apace,  etc.,  have  been 
included  In  the  "day"  coats. 

If  the.  experimenter  ie  willing  to  pay  S  dollars  for  one  assay  of 
the  material,  then  the  conjunctive  use  of  the  variance  and  the  coat 
function  will  optimise  the  assay  procadura  for  that  coat.  Convaraaly, 
the  coat  function  will  show  him  what  ha  would  naed  to  pay  to  obtain 
an  Assay  for  a  prescribed  precision.  The  optimisation  processes  are 
shown  in  the  next  section. 

III.  OPTIMIZATION  PROCESSES. 

A.  Lagrangian  Multipliers 

Let  us  write  the  following  function 

L  -  V(X)  +  \S 


which  includes  both  the  variance  and  the  cost  as  a  quantity  to  minimize. 
Following  tha  standard  procadure,  partial  darlvativea  of  tha  function  L 
arc  taken  with  respect  to  the  parameters  d,  c,  p,  r  giving  four  equations 
in  five  unknowns,  the  fifth  unknown  being  the  variable  X .  The  fifth 
equation  is  the  cost  function  itself.  These  derivatives,  which  can  be 
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seen  in  Lite  Appendix,  give  rise  to  s  set  of  five  non—  n™»r  simultaneous 
equations  whose  solutions  are  also  shown  in  the  Appendix  and  were  ob¬ 
tained  as  a  unique  set  by  virtue  of  the  simplicity  of  the  nested  design. 

It  is  clear,  although  not  proved  mathematically,  that  any  hierarchical 
design  with  random  effects  and  an  associated  nested  cost  function  will  have 
these  two  properties:  non-linearity  and  analytic  solution. 

The  procadure  involving  Che  Lagrangian  multipliers  clearly  treats 
d,  c.  p,  r  as  continuous  variables.  Obviously  they  must  bs  positive  as 
well  as  all  of  the  other  input  values  such  as  cost  rates  and  the  variance 
components.  In  keeping  with  this  approach,  we  should  then  expect  to  find 
the  solution  for  these  variables  to  be  non-integers.  For  a  limit  of 
S  -  $9.00  per  assa;',  the  optimum  solution  gave 

d  -  1.98;  c  -  .79;  p  -  1.73;  r  -  3.54;  V(X)  -  7.4;  S  -  9.0 

Because  the  experimenter  must  work  with  integer  values  of  these 
variables,  the  next  highest  and  lowest  integer  value  of  each  variable 
were  examined  in  combinations  with  similar  upper  and  lower  values  for 
all  other  variables  giving  rise  in  general  to  2*  combinations.  The 
variance  and  cost  functions  were  computed  for  8  of  the  16  combinations 
plus  one  other;  and,  the  one  that  minimised  the  variance  for  the  allotted 
cost  was  chosen:  specifically  where  S  ••  $9.00,  d  ■  2,  c  -  1.  p  ■  1,  r  •  5. 
These  9  combinations  are  shown  in  the  table  below  with  the  selected  set 
shown  with  an  asterisk. 


d 

a 

2 

jr 

m. 

S 

1 

1 

1 

3 

17.0 

4.1 

1 

1 

1 

4 

15.8 

4.3 

1 

1 

2 

3 

13.0 

5.2 

1 

1 

2 

4 

12.4 

5.6 

2 

1 

1 

3 

8.5 

8.2 

i 

Si 

1 

1 

4 

7.9 

8.6 

2 

1 

2 

3 

6.5  . 

10.4 

2 

1 

2 

4 

6.2* 

11.2 

2 

1 

1 

5 

7.5 

9.0* 

By  extending 

the  perturbation  range 

beyond  adjacent  values,  such 

a  table 

also  has 

the  advantage  of  displaying  to 

the  experimenter  what 

a  small 

increase 

in  cost 

would  yield 

in 

terms  of 

Increased  precision. 

Similar 

solutions 

would  be  available 

at 

ocher  levels  of  possible  cost 

S,  per  assay. 

B.  Integer  Programming 

Integer  programming  is  that  type  of  mathematical  programming  that 
seeks  to  optimize  a  particular  function  subject  to  certain  constraints 
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A ppffvntv 


(1)  S  *  Dd  +  Cdc  +  Pdcp  +  Rdcpr 

a3  a3 

(2)  L  -  [V(X)  +  A(S)]  - - + 


d  dc  dep  dcpr 

-  AS  +  ADd  +  ACdc  +  APdcp  +  ARdcpr 
_  2 


(3)  A 


A  -  r 

n  j  2  2  2  2 

Rd  c  p  r 

-0 

oL  _  p 

+  A[Pdi 

2 

“  °r 

3p 

3L  1 

dcp2 

-°c2 

.  2 
dcp  r 

zi!+ 

3c  1 

i 

‘  2  + 

dc 

.  2  + 
dc  p 

2 

dc  pr 

i 

_  2 

2 

2 

3L 

■  —V’ + 

I!c_  + 

“°P  . 

3d”  ; 

d2 

d2c 

p  T 

d  cp 

(7)  t 


(8)  p2  - 


(9)  c2.  -f. 


(10)  d 


D  +  C* 


*r+'i-r'~ '+r 


(11)  d 


Vd"  •  ad  .  S 


D(  0dVT  +  ocVc"  +  o  Vp”  +  CfV7 
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AN  APPLICATION  OF  MATHEMATICAL  PROGRAMMING  TO  EXPERIMENTAL  DESIGN 


u.  Richard  Moore 

U.  S.  A.  Aberdeen  Research  and  Development  Center 
Aberdeen  Proving  Ground,  Maryland  21005 


It  often  happens  that  a  number  of  observations  (measurements)  of  Interest 
can  be  made  each  time  a  complex  system  is  operated.  When  there  are  seyeral 
different  modes  in  whici  such  a  system  can  be  operated  (missions)  the  set  of 
observations  may  be  different  for  each  mission.  Many  observations  may  be 
common  to  several  modes  of  operation  while  some  may  be  peculiar  to  a  particular 
mission. 

i 

*  When  it  is  necessary  to  obtain  a  number  of  observations  for  each  of  a 
knowrf  set  of  subsystems -(or  functions)  th$  question  of  how  many  times  the  system 
must  perform  each  type  mission  arises.  A  reliability  demonstration  test  for  a 
complex  system  Is  an  example  of  the  above.  Tt)j^  paper  glvis  an  example  of  a  , 
procedure  by  which  the  required  set  of  observations  can  be  obtained  at  a 
minimum  cost.  Hopefully  this  example  will  help  those  who i  are  active  in  research 
and, development  to  recognise  situations  In  which  appreciable  saving  can  be 
effected  by  the  use  of  mathematical  programming.  These. procedures  are  relatively' 
simple  to  use  once  the  problem  to  which  they  apply  has  been  Identified. 

i  r 

As  an  example,  consider  a  complex  system  which  consists  of  six  major  sub¬ 
systems.  A  reliability  demonstration  test  must  be  conducted  for  the  system  and 
also  for  each  of  the  subsystems  operating  as  part  Of  the  system.  This  system  Is 
capaWe  of  performing  four  different  types  of  mission.  The  amount  of  operating 
time  (number  of  operating  cycles)  for  each  subsystem  during  a  mission  depends  on 
the  mission  type  but  it  is  assumed  that  the  probability  that  a  specific  subsystem 
will  fa^i  during  a  mission  depends  only  on  the  amount  of  operating  time  (cycles) 
It  accumulates  during  the  mission  and  not  on  the  type  of  mission  which  the  system 
is  performing.  That  Is,  it  Is  assumed  that  the  distribution  of  time  (cycles)  to 
failure  does  not  depend  on  the  type  of  mission  the  system  Is  performing  when  the 
subsystem  Is  accumulating  operating  time  (cycles). 


This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 
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It  is  necessary  to  use  a  reliability  demonstration  plan  which  allows  no 
failures  because  of  the  limited  availability  of  test  resources.  For  the 
system  to  De  tested  tnis  dictates  that,  in  order  to  demonstrate  tne  desired 
reliability  with  the  required  confidence,  each  of  the  subsystems  must  accumulate 
the  operating  experience  given  in  Table  I. 


TABLE  I 


Required  Subsystem  Operating  Experience 


Subsystem 

Operating  Time  (Cycles) 

1 

60  minutes 

2 

30  minutes 

3 

40  inlnutes 

4 

80; mi nutes 

5 

20  cycles 

6 

30  cycles  j 

i 

It- is  further  required  that  the  system  must  complete  at  le^st  two  of  each  type 
mission  without  failure.  P  '  \  l  <J 

The  operating  time  (cycles)  for  each  subsystem  during  eachtype  mission  1ST" 
given  in  Table  II  ' 

TABLE  II 

Operating  Experience  for  Subsystems  by  Mission  Type 


Subsystem 

Mission  Tyi 

pe 

Z 

3 

4 

1 

6 

12 

0 

3 

2 

9 

0 

4 

6 

3 

8 

2 

0 

5 

4 

0 

6 

10 

3 

5 

5 

1 

2 

0 

6 

3 

0 

2 

3 
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The  units  of  measurement  for  the  entries  in  Table  II  are  minute^  for  subsystems 

/ 

one  through  six  and  cycles  for  subsystems  seven  and  eight.  Thus,  Table  II  says 
that  a  subsystem  one, operates  for  six  minutes  during  a  Type  1  mission,  twelve 
minutes  during  a  Type  2  mission,  three  minutes  during  a  Type  4  mission  and 

i  „ 

doesn't  operate  during  a  Type  3  mission,  etc. 

j.  The  problem  is  to  find  the  best  v/ay  to  run  this  test  program  in  the  sense 
that  the  total  cost  of  testing  is  minimized.  The  cost  of  running  each  type 
mission  is  given  in  Table  III. 


TABLE  III 

Cost  of  Running  Each  Type  Mission 


The  cost  data  In  Table  II  are  relative  costs  which  are  in  thousands  of  dollars 

In  this  example.  x-~. 

,  / 

With  the  preceding  information  available,  it  Is  seen  that  this  is  a  problem 
in  linear  programming  In  which  one  must  determine  the  minimum  number  of  times  to 
require  the  system  to  complete  each  type  mission,  1.e.,s  minimize  the  fcost  of 
testing,  subject  to  the  constraints  that  each  subsystem  must  accumulate  at  least 
much  operating  experience  as  1,s  given  In  Table  I  and  also  the  system  must 
complete  at  least  two  of  each  type  mission.  Since  the  solution  will  be  In  terms 
of  number  of  tests,  It  also  follows  that  the  solution  vector  for  the  linear 
program  must  be  integer  valued.  A  problem  of  this  type  Is  called  an  integer 
programming  problem. 

The  first  step  in  finding  a  solution  to  the  problem  will  be  to  formulate  it 
in  standard  terminology.  Let  Y  be  a  column  vector  with  jth  element  equal  to  the 
number  of  times  the  jth  type  mission  must  be  run,  that  Is 

Yi 

Y  - 
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where  iy^ I  are  unknown  which  must  be  determined.  Let  C  be  a  row  vector  with 
elements  equal  to  the  mission  costs  given  in  Table  III,  B  be  a  column  vector 
with  elements  equal  to  the  required  total  operating  requirement  for  each 

tpm  a*.  given  in  Tahlft  T  *nH  A  ho  a  matniy  wHth  SlfifTSHtS  0CJ*J2l  tO 

operating  tin/.1  (cycles)  for  each  mission  type  as  given  in  Table  II.  With  this 
notation  the  problem  is  to  find  the  vector  Y  such  that: 

CY  -  minimum 
AY  >  B 

Y  > 

and  all  of  the  elements  of  Y  must  be  integers.  Substituting  the  numbers  given 
in  Table  I,  Table  II  and  Table  III  in  the  above  relationships  leads  to  the 
following  statement  of  the  problem: 

Find  a  vector  of  integers,  Y,  such  that 
CY  =  48y !  +  39y2  +  43y3  +  39y4  =  minimum 

and 

6y  i  +  1 2y ;  +  3y4  >_  60 

9y  i  +  4y3  +  6y,t  >.  30 

8y,  +  2y^  +  5y„  >  40  f 

A  •  Y  -  6y;,  +  1 0y +  3y i,  >  SO  !  B  ’  1 

5yj  +  y2  +  2y3  >  20 

3y i  +  2y?  +  3y(f  >_  30 

yi  n  2 

Y  =  y?  2 

y  — 2 
y ,  _  2 
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The  last  four  constraints  can  be  eliminated  from  the  problem  by  the  translation 
of  axis  i 


This  transforms  the  original  problem  to  the  equivalent  problem:  Find  a  vector  X 
such  that 

CX  ®  minimum 
AX>Sx 

X  >  0 

where  Bx  e  B  -  AM 

ahd  M  *  2e  for  this  problem.  1 

The  next  step  is  to  inspect  the  vector  Bx  to  determine  whether  the  complexity 
.of  the  problem  can  be  reduced <  If  any  element  of  Bx  is  zero  or  negative  the  operating 
requirements  for  the  corresponding  subsystem  will  have  been  satisfied  when  the  system 
has  completed  the  number  of  each  type  mission  necessary  to  satisfy  the  constraints 
Y  >_  M.  Thus,  t.he  size  of  the  problem  can  be  reduced  by  dropping  all  constraints 
corresponding  to  zero  or  negative  elements  of  Bx. 

In  the  example  Bx  *  B  -  2Ae  is 
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The  second  element  in  Bx  is  negative,  which  means  that  if  each  type  mission  is 
run  two  times  in  order  to  satisfy  the  requirements  X  >_  0  (Y  >_  2e)  there  will  be 
(9)(?)  +  (Q)(2)  +  (4 ) ( 2 )  +  (6)(2)  a  38  minutes  operating  time  on  subsystem  number 
two,  which  is  eight  minutes  more  than  the  required  thirty  minute  operating  time 
for  that  subsystem;  hence,  the  second  element  in  Bx  is  minus  eight.  This  allows 
a  reduction  in  the  size  of  the  problem  without  changing  the  solution  by  dropping 
the  second  constraint  equation. 

Letting  AQ  and  B°  represent  the  reduced  matrix  and  vector  resulting  from 
the  deletions,  the  problem  Is  rewritten  as: 

Find  a  vector  X  such  that 

CX  =  minimum 
V  2.  B°x 
X  >  0 

The  next  step  is  to  inspect  the  columns  of  the  matrix  Aq  to  determine  whether 
the  dimension  of  the  problem  can  be  reduced  by  omitting  one  or  more  of  the  elements 
o*'  the  vector  X.  If  any  column  of  AQ  contains  all  zeros  then  none  of  the  subsystems 
for  which  data  is  needed  will  operate  during  the  type  mission  corresponding  to  that 
column..  Requiring  the  system  to  complete  such  a  mission  would  only  increase  the  cost 
of  testing  without  generating  anyj needed  data.  The  dimension  of  the  problem  is 
reduced  by  eliminating  the  zero  columns  from  AQ  and  the  corresponding  elements  from 
the  vectors  x  and  C.  Letting  A°,  X°  and  C°  be  the  new  matrix  and  vectors  which  result 
from  this  transformation,  thej  problem  Is  stated  in  final  form  as:  Find  a  vector  X° 
such  that  ! 

i  C°X°  s  minimum 


the  second  row  In  A  and  the  second  element  in 


*  minimum 


AeX°  >  B°x 
X6  >  0 

In  the  numerical  example,  omitting 
Bx  reducer,  the  problem  to  the  form: 


CX  =  [40 


38 


43  39) 


*2 

*3 


l  *4  / 
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The  dimension  of  the  proper,  can  not  be  reducej  because  none  cf  the  columns  of  Aq 
contain  oni ,  zero  eluents.  Thus,  it  is  in  the  final,  reduced  fern. 

With  ‘Jo  exception  of  the  requirement  that  the  elements  of  V  must  be  integers, 
the  above  is  simply  a  linear  programming  problem  wine-  can  be  solved  using  the 
simplex  method.  It  can  be  shown  that,  if  all  of  the  elements  of  A0  are  either  zero 
or  one  and  all  of  the  elements  of  8®  are  integers,  then  the  solution  vector  obtai^d 
using  the  simplex  method  will  have  all  integer  elements.  This  would  be  the  case  ifl^ 
each  of  the  subsystems  were  of  cyclic  nature,  e.g.,  relays  which  operate  for  only  one 
cycle  when  required  during  a  mission.  Also,  if  for  each  subsystem,  the  operating 
time  (number  of  cycles)  is  the  same  during  each  mission  in  which  it  is  required  to 
operate  and  the  operating  requirement  is  a  multiple  of  this,  then  the  problem  can  be 
reduced  to  the  zero-one-integer  problem  and  the  simplex  method  can  be  used  to  find 
the  solution  vector  for  the  Integer  programming  problem.  It  should  be  noted  that  the 
above  remarks  apply  to  the  problem  after  it  has  been  put  in  reduced  form  by  making, 
all  possible  decreases  in  size'and  dimension.  That  is,  even  though  it  is  not  possible 
to  formulate  the  original  problem  as  a  zero-one-integer  problem  it  may  be  possible  to 
do  so  for  the  problem  in  reduced  form. 

It  should  be  emphasized  that  the  conditions  of  the  preceding  paragraph  are 
sufficient  but  not  necessary  to  assure  that  the  simplex  method  will  yield  an  integer 
solution  to  the  problem.  Since  the  simplex  method  is  much  easier  to  apply  than  the 
methods  of  integer  programming,  it  is  advisable  to  first  find  the  simplex  solution  and 
see  whether  it  is  integer  valued  before  proceeding  further. 


521 


The  simplex  solution  for  the  example  given  here  is  not  integer  valued  so  a 
search  routine  was  used  on  the  computer  to  find  the  solution,  which  is: 


■  l°\  I2) 

I  J  ! 

X  =  3  or  Y  =  X  +  2e  =  5  ; 

\  3  '  \  5  ; 


with  minimum  cost  of 

2  \ 

3  | 

CY  =  (40  38  43  39)  *  g  »  604 

5 

The  reader  may  find  it  interesting  to  show  that  the  solution  satisfies  all  of  the 
constraints. 

The  problem  formulated  here  is  of  the  same  structure  as  the  nutrition  or  diet 
model  found  in  the  literature.  The  reader  is  referred  to  Chapter  27* of  "Linear 
Programming  and  Extensions"  by  George  B.  Dantizg  for  an  excellent  treatment  of  the 
formulation  and  treatment  of  that  problem. 
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Transmission  of  Infraaonlc  Ware a 
Generated  by  Large  Miaaile  Launebea 

by 

R.  1.  Lacy  and  C.  X.  Sharp 
Institute  for  Exploratory  Research 
U.  8-  Army  Electronics  Coseeand. 
Fort  Monmouth,  lev  Jeraey 


Infraaonlc  pressure  waves  generated  during  launching  of  large  als- 
siles  can  be  detected  at  distances  greater  than  2000  kilometers.  On  a 
number  of  occasions  during  recent  years ,  two  infraaonlc  wave  trains 
spaced  about  30  alnutes  apart  have  been  recorded  by  sensor  array  areas 
located  at  Fort  Monmouth,  I.  J.  These  areas  are  approximately  1400  kil¬ 
ometers  from  the  launch  pads  at  Cape  Kennedy,  Florida.  Vo  date,  no 
explanation  for  these  observed  early  arrivals  of  an  infraaonlc  wave 
train  which  appears  to  travel  at  approximately  twice  the  normal  veloc¬ 
ity  of  aound  through  the  atmosphere,  has  proven  to  be  acceptable.  Most 
recently,  ve  have  observed  a  relationship  between  these  sounds  and  the 
Jet  streams  traveling  the  proximity  of  the  path  from  Cape  Kennedy  to 
Fort  Monmouth.  Oh  this  basis,  we  have  evolved  a  hypothesis  that  such 
correlation  exists  between  the  Jet  streams  end  this  anomalous  propaga¬ 
tion  of  sound  waves.  Because  of  the  large  amount  of  Jet  stream  data, 
but  relatively  few  missile-firing  events,  s  statistical  dstlgn  of  fur¬ 
ther  experiments  adequate  to  test  the  hypothesis  is  planned.  A  theo¬ 
retical  analysis  explains  the  pressure  wave  spectra  received  aa  con¬ 
sisting  of  three  separate  groups  of  spectra. 

Introduction 

T-  '  —  "■  — — 

Infraaonlc  wave  propagation  phenomena  have  been  explored  with  re¬ 
newed  Interest  in  recent  years.  In  this  report  we  euamarlse  our  cur¬ 
rent  findings.  Ve  are  concerned  here  only  with  infraaonlc  frequencies 
traversing  the  atmosphere.  Infraaonlc  waves  are  inaudible  sound  waves 
Whose  frequency  of  oscillation  are  below  1$  Hz;  they  have  the  same 
velocity  as  audible  sounds,  l«e<,  about  332  m/aee  at  20°c  sad  a  pres¬ 
sure  of  760  amlg.  The  absorption  of  inf  mound  is  considerably  lass 
than  absorption  of  audible  sound  of  the  atmosphere  dus  to  hast  conduc¬ 
tion  sad  viscosity.^  Because  of  this,  the  detection  at  long  distances 
of  infraaonlc  energy  generated  by  the  large  missiles,  as  represented 
by  Titan  III  and  Saturn  V,  launched  from  Capa  Kennedy,  Florida,  is  pos¬ 
sible.  Frequency  analysis  of  a  number  of  magnetic  tape  recordings  mads 
at  Fort- Monmouth,  X.  J.  during  tbe  arrival  of  infraaonlc  waves  generated 
by  the  missile  launches  show  the  maximal  energy  propagated  to  be  can¬ 
tered  within  0.5  to  5  !*•  The  maxima  of  these  traveling  pressure  oscil¬ 
lations  recorded  in  the  Fort  Monmouth  test  areas  seldom  exceed  2.5 
dynes /car .  Tbs  ''normal"  traveltime  for  infraaonlc  waves  from  Cape  Ken¬ 
nedy  to  Fort  Moamouth  is  between  JO  to  6$  alnutes  after  launch.  This 

This  article  has  been  reproduced  photographically  from  the  author's 
manuscript. 
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variation  la  arrival  time  may  be  dua  to  the  directions  mad  velocities 
of  tha  surface  triads  aloof  the  travel  path*  The  duration  of  the  lnfrs- 
aonlc  wave  trains  also  baa  baan  observed  to  vary  between  five  sad  ten 
al«ui«s>  Inis  variation  aay  ne  tbe  result  of  tbe  relative  signal-to- 
noisa  ratio  at  tbe  sensor  location  for  each  launching,  furthermore  a 
so-called  precursor  or  vave  train  baa  baan  tpo radically  observed  that 
arrived  about  10  minutes  earlier  than  tbe  "normal"  wave  train.  Figure  1 
clearly  above  this  precursor  recorded ,  vban  the  wind  turbulence  was 
practically  nil/  at  a  site  1000  ka  north  of  Cape  Kennedy/  Florida. 

The  Wind  Turbulence  Frobleai 


Tbe  detection  at  long  distances  of  acoustical  energy  produced  by 
tbe  large  niasila  launches  (Titan,  Atlas,  Saturn,  etc. )  from  Cape  Ken¬ 
nedy  la  complicated  by  lev  slgnal-to-noise  ratio*.  Tbe  acoustical  en¬ 
ergy  to  be  detached  la  in  the  form  of  atmospheric  pressure  vaves  of 
usually  leas  than  2  dyn as/ca2,  contributed  in  the  selected  frequency 
spectra  0.01  to  10  Is.  At  tbe  distance  of  interest  for  this  study,  ap¬ 
proximately  lbOO  km,  tbe  amplitude  and  frequency  range  of  tbe  pressure 
vaves  detected  have  been  decreased  considerably  by  atmospheric  absorp¬ 
tion,  by  tbe  dispersive  effects  of  wind  and  temperature  variations  in 
tbe  lover  and  upper  atmospheric  regions,  end  by  spherical  spreading. 

At  long  ranges,  tbe  combined  effect  of  these  factors  results  in  a  reduc¬ 
tion  of  the  amplitude  of  the  missile-produced  atmospheric  pressure  waves 
to  less  than  the  random  local  atmospheric  pressure  variations,  ambient 
at  tbe  sensor  Location.  Theta  sab lent  variations  due  to  local  ataos-  > 

pherlc  conditions  can  have  pressure  amplitudes  in  the  order  of  100  to 
500  dynes /cm2  and  frequency  spectra  similar  to  aissile-produesd  vaves. 

tv 

We  found  tbe  most  promising  approach  used  to  overcome  these  dif¬ 
ficulties,  when  tbe  position  of  the  infraaonlc  source  is  known,  to  be  in 
tbe  employment  of  a  large  number  of  sensors  (microphones,  nicrobargraphs, 
or  pressure  transducers)  in  e  linear  array,  parallel  to  tbe  infraaonlc 
pressure  wave  front  and  tbeuseof  real-time  correlation  techniques. 

The  output  of  each  sensor  la  tbe  array  la  combined  with  tbe  outputs  of 
the  others  by  summation  or  multiplication.  If  the  sensors  are  suffi¬ 
ciently  spaced  in  the  linear  array,  tbe  pressure  variations  caused  by 
vial  turbulence  at  each  sensor  tend  to  produce  uncorrslated  outputs. 

These  uncorrelated  outputs  can  be  expected  to  sum  povsr-vlse  to  1A2, 
where  I  le  tbe  number  of  sensors  and  A  is  tbs  output  amplitudes;  vhareas 
tbe  missile-produced  pressure  vaves  arriving  in  phase  at  all  sensors  la 
the  broadside  linear  array  vlll  sun  up  to  (1A)2.  Tbe  improvement  In  tbe 
slgnal-to-noise  ratio  over  that  of  a  single  sensor  vlll  be  equal  to 
10  log  (MAfi/MA2  or  in  dJ  equal  to  10  log  I.  Tbe  greater  tbe  number  of 
sensors  in  tbe  array,  of  course,  tbe  greater  tbe  detection  Improvement2 
(Fig.  2).  In  addition,  an  effective  wind  screen  (Fig.  3)  has  been  de¬ 
vised  to  inclose  each  sensor.  This  scrssnlng  can  rasult  in  a  further 
improvement  in  signal-to-noise  ratio  of  10  to  30  dB.  Tbe  amount  of  im¬ 
provement  is  e  function  of  the  degree  of  vind  turbulence. 
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A  1000'  broadside  linear  array  of  20  screened  and  equally  spaced 
microphone  sensors  hare  been  in  operation  for  several  years  at  a  test 

iu  Veysid®;  s-  <j.  (jh #.  1*}.  t«  n >•  m  v— vH fy  th*  riiMption  a t 
arrival  and  determine  the  velocity  of  propagation*  a  second  linear  array 
of  10  sensors  is  in  operation  at  a  site  in  Middletown*  M.  J.,  seven  miles 
north  of  Wayside.  Both  arrays  are  arranged  parallel  to  the  acoustic- 
pressure  wave-fronts  arriving  from  Cape  Kennedy,  these  arrays  are  re¬ 
motely  controlled  and  the  outputs  recorded  via  telephone  circuits  ter¬ 
minating  in  a  laboratory  room,  the  time  of  pressure  wave  arrival  from 
launching  time  T0*  the  wave  durations*  velocity*  and  other  pertinent 
data  are  recorded  for  each  array  and  each  achedulsd  large  missile 
launching. 

Observations 

The  occasionally  observed  early  arrival  (precursor)  of  infrasonic 
waves  in  the  time  frame  of  38  to  43  minutes  (Pig.  l)  preceding  the  nor¬ 
mal  arrival  period  is  of  considerable  interest  to  those  concerned  with 
acoustic  propagation  phenomena.  Observed  amplitudes  and  durations  of 
these  early  arrivals  are  about  one-half  that  of  the  normal  arrival.  In 
the  lower  atmospheric  regions*  the  absorption  coefficient*  which  is  a 
function  of  temperature*  viscosity*  end  pressure*  is  extremely  small  at 
these  low  acoustic  frequencies.  In  the  upper  atmospheric  regions*  the 
absorption  coefficient  is  greeter*  principally  because  of  the  lower  at¬ 
mospheric  pressure.  This  lends  support  to  our  hypothesis  that  upper 
atmospheric  ducting  is  responsible  for  the  early  arriving  pressure  waves 
at  the  Jet  stream  level  (30  000  to  kO  000').  Apparently*  early  arrival 
occurs  only  when  the  Jet  stream  flows  northward  along  the  eaat  coast 
from  the  direction  of  Florida.  We  have  observed  this  phenomena  during 
the  winter  months  and  it  has  been  confirmed  by  studies  of  the  tropo- 
pause  wind  analysis  stream  function  charts  published  dally  by  the  B8SA 
Environmental  Data  Service. 3  The  velocity  of  the  Jet  stream  as  given 
in  these  charts  plus  the  velocity  of  sound  at  the  relevant  altitude 
could  account  for  the  early  arrival  of  the  Infrasonic  vave  train.  This 
theory  Is  further  confirmed  by  comparing  the  observed  reduced  vave  am¬ 
plitudes  and  durations  of  the  early  arrivals  with  the  amplitudes  and 
durations  of  the  normal  arrivals.  A  four-minute  film  showing  the  var¬ 
iation  of  thaae  wind  streema  over  a  period  of  a  year  will  accompany 
this  presentation.  These  data  show  that  the  Jet  streams  prevail  in  a 
vest  to  eaat  direction  across  the  U.S.A.  in  tbs  summer.  Aa  late  fall 
occurs*  the  streams  drop  to  ths  southern  portion  of  the  country  with  an 
attendant  curvature  to  the  north  along  the  eaat  coast.  Figure  5  shows 
typical  west  to  east  sunMrtias  patterns*  while  Pigurt  6  shows  the  win¬ 
ter  streams  approximately  parallel  to  the  east  coast.  The  latter  is 
thereby  in  the  correct  orientation  to  account  for  the  precursor,  or 
earlier  then  normal  wavs  arrival. 


525 


Dealga  of  Experiment-Statistical  Proto  1—  Areas 


During  observation!  of  the  early  infraeouad  arrivals  from  Caps  Ken¬ 
nedy  ■  the  detection  of  the  weak  infresonle  mm  trains  from  lone  dis¬ 
tances  is  often  aasked  by  the  effects  of  wind  turbulence ,  particularly 
during  the  windy  winter  souths  in  the  northeastern  part  of  the  U.  8.  The 
broadside  array  of  anay  sensors  {ICO  or  bo  re)  appears  to  be  the  aost 
promising  solution  to  the  reduction  of  the  effect  of  wind  turbulence. 

The  alcro phone  sensors  presently  eaployed  require  considerable  aalnton- 
ance  and  therefore  large  arrays  of  s  hundred  or  no  re  sensors  become 
rather  Impractical.  At  the  present  time,  the  problea  of  further  reduc¬ 
ing  the  effects  of  wind  turbulence  reaalns  with  us*  Investigation  of 
two  types  of  lnfrasonlc  gradient4  alcrophone  seniors  are  currently  in 
progress.  The  gradient  types  offer  promise  of  greater  reliability,  lower 
costs  per  unit,  and  nay  be  less  affteted  by  wind  turbulence. 

Then  the  problea  of  site  selection  for  the  sensors  further  broadens 
the  experimental  design  problea.  With  but  only  two  or  three  site  instru¬ 
mentations  economically  allowed,  the  question  arises  as  to  where  the  sea-  • 
pling  would  prove  to  be  aost  profitable  in  terns  of  helpful  data.  Addi¬ 
tionally,  the  correlation  of  the  sparse  experimented,  data  with  the  volumi¬ 
nous  wind  data  provide  problea  areas  which  nay  be  aaenable  to  statistical 
techniques.  As  shown  In  the  accompanying  film  and  Figures  5  and  6,  the 
wind  data  shows  distinct  auamar-vlater  variations.  But,  the  launching 
of  alssiles  are  relatively  few  end  far  between. 

To  further  complicate  the  real  situation,  there  could  be  •i4"' 
other  hypotheses  attempting  to  explain  the  relative  occurrences  of  the 
no  real  and  precursor  wave  trains.  For  this  experimental  design  consid¬ 
eration,  though,  we  are  confining  our  attention  to  the  one  hypothesis, 
i.e.,  the  Jet  stream  hypothesis.  We  plan  to  arrange  experiments  to  con¬ 
firm,  or  deny,  this  hypothesis.  Initially,  therefore,  we  Bust  date  rains 
how  best  to  proceed  in  this  endeavor  that  is  typical  of  aany  such  quests 
in  what  slight  be  terned  macroscopic  experimental  research  involving  spo¬ 
radic  sad  uncontrollable  conditions. 
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;  APPEBDIX  I 

/  Wave  Spectra  / 

A  theoretical  analysis  was  performed  in  an  attempt  to  explain  math¬ 
ematically  the  total  process  of  generation,  transmission*  and  reception 
of  these  pressure  waves  as  typified  by  Figure  1.  This  theoretical  ap¬ 
proach  ie  fundamental  In  nature  as  it  provides  understanding  of  the  char¬ 
acteristics  to  be  expected  from  signals  so  formulated.  It  is  planned  to 
present  this  original  and  extensive  theory  in  a  separate  publication  as 
yet  undetermined. 

Of  Interest,  though,  to  those  concerned  with  the  physics  is  that  a 
close  examination  of  the  experimentally  obtained  signals  of  Figure  1 
show  a  high  correlation  with  these  theoretical  findings.  Pronounced 
characteristics  that  are  readily  apparent  in  Figure  1  coincide  with  the 
analytical  results.  This  figure  shows  that  between  approximately  47  and 
53  minutes  from  launch-time,  the  normal  signal  consists  of  relatively 
prominent  groups  of  signal  spectra  having  amplitudes  clustered  at  por¬ 
tions  located  46,  49,  and  3^*7  minutes  from  launch  time.  It  may  be  fur¬ 
ther  noted  that  the  first  group  of  spectra  at  46  minutes  is  relatively 
smmll  in  amplitude,  while  the  second  two  groups  are  contrastingly  large. 

This  clustering  sad  relative  amplitudes  are  in  close  accord  with 
the  theoretical  findings,  which  show  an  expectation  that  three  major  time 
periods  of  energies  will  be  experienced  in  the  reception  of  such  pressure 
signals.  The  first  period  is  mathematically  predicted  to  consist  of 
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yig.  2.  An  example  of  the  correlated  output  from  seven  microphones  In  a  broadside  array. 


Experimental  infrasonic  broadside  array  tit  20  microphones. 


FIG.  5  TYPICAL  SUMMERTIME  JET  STREAM  PATTERNS 


FIG.  6  TYPICAL  WINTERTIME  JET  STREAM  PATTERNS 
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ABSTRACT 

This  paper  deals  with  the  probability  of  catastrophic  failure  and  target 
miss  of  an  antitank  missile  in  flight  resulting  from  failure  of  control  jets  to  fire. 
When  two  failures  occur  in  sequence,  the  missile  deviates  widely  from  the  flight 
path  but  is  recoverable.  When  three  jets  fail  to  fire  in  certain  sequences,  cata¬ 
strophic  missile  failure  will  result. 

A  complete  solution  to  this  problem  would  involve  the  determination  of  the 
probability  of-  failures  of  all  possible  combinations  of  successes  and  failures 
which  would  result  in  ground  impact.  It  would  consider  the  random  distribution 
of  the  location  of  the  target  relative  to  launch  position  and  the  distribution  of  the 
missile  about  an  average  flight  path. 

A  partial  solution  to  this  problem  has  been  found  in  a  system  of  "states, " 
or  intervals  in  the  vertical  plane.  The  recursive  equations  for  the  probability  of 
lying  in  each  state  have  been  developed  and  tables  of  state  probabilities  for 
several  values  of  the  probability  of  success  for  one  jet  pair. 


INTRODUCTION 

The  specific  problem  addressed  in  this  paper  deals  with  the  probability 
of  failure  for  a  missile  which  employs  jet  pulses  as  a  control  mechanism  and  as 
a  means  for  overcoming  gravity. 

In  this  system  pairs  of  jet  pulses  fire  sequentially  as  the  missile,  which 
maintains  a  fairly  constant  roll  rate,  flies  toward  the  target.  The  pairs  of  jet 
pulses  are  so  located  about  the  center  of  gravity  that  little  or  no  net  torque  is 
applied  to  the  missile  body  (Figure  1) .  The  firing  position  of  the  jets  is  in  a 
near  vertical  plane.  By  varying  the  angle  away  from  the  vertical  plane,  a  side 
component  of  thrust  may  be  attained.  This  side  force  makes  lateral  corrections 
in  the  flight  path  upon  command  (Figure  2) . 


This  article  has  been  reproduced  photographically  from  the  authors 
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Figure  1.  Location  of  Jets  (Side  View) 


The  major  concern  is  with  the  effect  on  the  flight  trajectory  of  a  pair  of 
jet  pulses  failing  to  fire  at  the  correct  time.  Since  the  missile  is  acted  on  by 
gravity,  it  will  drop  immediately  unless  continually  sustained  by  these  jet  pulses. 
The  lateral  dispersion  will  be  affected  also;  this  may  or  may  not  be  serious 
depending  on  the  location  of  the  target  with  respect  to  location  of  missile  when 
failure  occurs  and  on  the  magnitude  of  such  disturbing  forces  as  cross  wind.  In 
this  study,  dispersion  in  the  vertical  plane  alone  is  dealt  with. 

The  missile  is  of  a  type  which  is  continually  commanded  from  launch  to 
target.  In  this  paper,  it  is  assumed  that  failure  problems  do  not  exist  in  the 
guidance  system,  but  occur  only  in  the  control  mechanism. 

So  that  the  missile  can  be  brought  back  to  original  flight  path,  the  rate 
of  firing  increases  as  errors  become  larger.  The  time  interval  between  jet 
firings  is  a  discrete  fraction  of  nominal  firing  intervals.  The  nominal  firing 
rate  will  just  balance  gravitational  forces,  but  a  more  rapid  firing  rate  will 
force  the  missile  to  move  upward. 
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During  the  flight  test  program  a  known  rate  of  failure  has  been  observed. 
The  failures  have  been  in  the  circuitry  so  that  either  >>oth  jets  of  a  pair  fire 
simultaneously  or  Ijoth  fail. 

In  the  flight  test  program  it  has  been  observed  that  out  of  a  total  of  M 
commands,  L  pairs  have  failed  to  fire.  The  estimated  failure  rate  of  the  cir¬ 
cuitry,  therefore,  has  been  L/M.  There  is  no  reason  to  believe  that  the  rate 
will  change  unless  the  quality  is  improved  at  some  additional  cost  or  that  the 
circuit  is  redesigned. 

The  effect  on  the  trajectory  has  been  investigated  when  combi  nations  of 
failures  have  occurred  in  the  flight  program.  Kvcn  though  there  has  been  a 
fairly  high  rate  of  component  failure  in  the  flight  test  program,  the  missile  has 
not  hit  the  ground.  Although  the  missile  exceeded  a  desirable  control  band  for 
an  interval  of  time,  it  would  have  missed  the  target  only  if  the  failure  had 
occurred  immediately  before  impact. 

For  this  investigation  two  types  of  failures  have  been  defined: 

1>  The  missile  deviates  from  the  linc-of-sight  to  the  target  by  more 
than  one  unit.  In  this  case  it  exceeds  a  desirable  control  band 
which  will  result  in  a  miss  if  the  deviation  occurs  Just  prior  to 
impact. 

2)  The  missile  deviates  downward  three  units  and  will  impact  the 

ground.  This  case  is  a  catastrophic  failure  and  can  never  recover 
or  hit  the.  target. 


COMBINATIONS  OF  FAILURE 

The  first  approach  considered  for  this  problem  was  that  of  determining 
the  probability  of  certain  combinations  of  failure. 

If  it  is  assumed  that  there  are  N  jet  pulse  pairs,  each  with  probability  p 
of  success  and  corresponding  probability  of  failure  q  1  -  p  and  that  the  pairs 

fail  independently  of  one  another,  the  probability  of  no  failures  will  lx1  p‘\  The 
following  recursive  equations  hold: 

1)  Probability  of  no  repeated  failure  (FFi  in  a  sequence  of  N  (Table  li 
AN  =  pA\-i  4  ,k,AN-2’  A»  ’•  Ai  1  • 
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In  a  system  <>f  N  consecutive  independent  subsystems,  each  with  probability  p  of  success 
probability  of  no  refloated  failures. 
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Table  1 


21 


mi  i 


•  C1'  r 


Probability  of  at  least  three  failures  wi  a  ■  I  II 
of  N  =  (  Table  i) 


N 


pCN-l  '  <B’CN-3 


3)  Probability  of  at  least  one  pair  of  failure  l  IT  but  iu>  Mib.-,c<|ii<  nee 
with  three  or  more  failures  in  a  row  (Table  3i 


W.  =  1  -  A_  -  C 


N 


N 


N 


Trajectory  simulation  implies  that  when  any  combination  of  three  out  of 
four  consecutive  pairs  or  three  out  of  five  consecutive  pairs  of  jets  fail  to  fire  in 
the  right  sequence,  catastrophic  failure  results.  Some  combinations  of  three 
failures  in  six  consecutive  pairs  result  in  marginal  flight.  A  sequence  of  seven 
where  the  first,  seventh,  and  any  one  other  pair  in  between  fail  will  not  result  in 
catastrophic  failure.  In  fact,  if  every  fifth  jet  pair  failed  to  fire,  the  missile 
would  recover  but  the  dispersion  about  the  desired  flight  path  would  be  large. 

This  approach  was  abandoned  since  the  important  missile  position 
could  be  found  only  by  taking  a  particular  combination  of  failures  and  running  a 
trajectory  simulation.  The  approach  did  not  provide  direct  information  on  the 
probability  of  missile  failure.  For  example,  the  missile  would  fail  catastrophi¬ 
cally  if  failures  and  successes  alternated  (SFSFSF. . . ) ,  but  this  case  would  be 
included  in  the  calculation  of  A^. 


DISCRETE  STATE  STOCHASTIC  MODEL  OF  VERTICAL  DEVIATIONS 

The  next  approach  taken  involved  a  simplified  model  of  the  vertical 
deviation  from  an  average  or  nominal  trajectory. 

The  average  trajectory  was  computed  by  a  least  squares  fit  of  the  tra¬ 
jectory  when  all  jets  are  firing  in  the  proper  sequence  and  at  expected  time 
intervals.  The  least  squares  fit  established  an  average  or  expected  trajectory 
about  which  the  missile  oscillated  (Figure  3) .  In  making  the  fit  to  the  average 
trajectory  the  data  immediately  following  the  time  when  the  failure  occurs  were 
discarded.  The  data  were  again  used  when  the  firing  rate  once  more  became 
normal. 

At  the  time  of  firing  of  the  jet  pair,  the  missile  can  be  considered  in 
one  of  seven  "states, "  Each  state  is  an  interval  in  the  vertical  plane. 
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In  a  system  of  N  independent  consecutive  subsystems,  each  with  probability  P  <*f  success,  C^,  is  the 
probability  of  at  least  three  failures  in  a  row. 
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DEVIATION 


Figure  3.  Vertical  Deviation  from  Average  Flight  Path 


If  there  are  do  jet  failures,  each  pair  will  fire  when  the  missile  is  in 
state  0  (Figure  4).  The  missile  is  considered  to  be  in  state  0  when  the  first 
pair  is  fired.  Let  xo(  N)  denote  the  probability  of  being  in  state  0  at  the  time 

of  firing  for  Nth  pair,  then  X0(1)  =  1. 


When  the  first  failure  occurs,  the  missile  will  drop  into  state  2  (Figure  5). 
If  the  failure  is  followed  by  a  success,  the  thrust  of  the  jet  pair  does  little  more 
than  arrest  the  missile  in  its  fall;  thus,  the  missile  remains  in  state  2.  The  next 
success  will  bring  the  missile  to  state  1  and  the  next  success  brings  the  missile 
to  state  0.  For  example,  for  the  sequence  S,S,  F,S,S,S,  the  states  will  be 
0.0, 2, 3, 1,0. 


b<>2 


STATE  0 


Figure  5.  Vertical  Deviation  (One  Jot  Failure) 


Two  jet  failures  in  a  row  after  a  series  of  successes  will  drop  the  missile 
into  state  4  (Figure  6).  If  the  next  firings  are  successes,  the  missile  will  stay 
in  state  4  one  time  and  then  climb  up  one  state  at  a  time.  Thus,  the  sequence 

5.5, F,F,S,S,S,S,S,  will  result  in  the  states  0,0, 2, 4, 4, 3, 2, 1, 0.  For  the  sequence 

5.5, F,S,  F,S,S,S,S,S,S,  as  shown  in  Figure  7,  the  state  sequence  is 

0,0, 2, 2, 4, 4, 3, 2, 1,0. 

Three  failures  in  a  row  will  place  the  missile  in  state  6  (Figure  8) .  This 
is  a  captive  state  representing  the  catastrophic  failure  of  hitting  the  ground. 

Once  in  state  6,  the  missile  remains  in  this  state. 

The  recursive  equations  for  this  model  are  given  in  Table  4.  Here  XI(N) 

is  the  probability  of  being  in  state  1  at  the  time  of  the  N1*1  pair  firing.  These 
were  developed  through  the  following  type  of  reasoning:  The  missile  can  be  In 


DEVIATION 


TIME 


Figure  6.  Vertical  Deviation  (Two  Jet  Failures) 
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STATE  » 
STATE  i 
STATE  I 
STATE  4 


1  STATE  0 


Figure  7.  Vertical  Deviation  (Failure-Success-Failure  Sequence) 


DEVIATION 


Figure  8.  Vertical  Deviation  (Three  Jet  Failures) 
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previous  firing  and  that  firing  was  successful.  Thus,  Prob  (state  0  at  time  N' 
=  Prob  (statp  0  at  time  N  -  li  •  Prob  ( success )  +  Prob  (state  1  at  time 
N  -  1)  ■  Prob  (success),  or  XO(Ni  =  pX0(N  -  1)  ■*  PX1(N  -  1).  It  is 
assumed  that  the  probability  of  success  is  independent  of  the  state.  The  initial 
conditions  for  this  set  of  recursive  equations  are  given  in  Table  5.  The  values 
of  the  state  probabilities  for  N  =  1,  ....  25  and  p  =  .  50,  .  90.  .  05,  .  99  are 
shown  in  Tables  6  through  9. 


N.  N 
Statess'N^ 

1 

2 

3 

4 

5 

<; 

0 

n 

n 

P? 

P3 

p*  +  p3q 

p6  +  2p4q 

1 

p2q 

P3q 

p4q 

2 

q 

2pq 

2p2q 

2p3q 

2p4q  +  2p3q2 

3 

2pV 

4p3q3 

4 

q2 

3pq2 

4pV 

4pV 

5 

2PV 

6 

■ 

■ 

Hi 

4PQ2  +  q* 

q5  4  5pq4  •*.  gpJq3 

Table  5. 


The  probability  of  catastrophic  failure  (hitting  the  ground)  is  the  prob¬ 
ability  of  being  in  state  6.  If  a  target  were  of  such  a  size  as  to  cover  states  0, 

1,  and  2  exactly,  the  probability  of  hitting  the  target  would  be  the  probability  of 
being  in  any  of  these  three  states . 

There  are  several  drawbacks  to  this  approach.  The  target  might  not 
cover  an  exact  number  of  states  so  that  the  distribution  within  states  is  of 
importance.  The  range  of  the  target  should  be  considered  to  be  variable:  thus, 
the  time  between  "times  of  firing"  needs  to  be  taken  into  account.  As  the  pairs 
are  fired,  the  number  of  available  jet  pairs  decreases.  Thus,  if  a  failure 
occurs  near  the  end  of  flight,  the  time  until  another  unused  pair  is  in  position  to 
fire  is  longer  than  It  would  be  at  the  beginning  of  flight  and  the  missile  might 
drop  further.  For  this  reason,  the  model  does  not  provide  a  uniformly  good 
approximation  to  reality. 
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One  of  the  limitations  of  this  approach  is  the  difficulty  of  relating  the 
position  of  the  missile  in  one  of  the  various  states  to  its  position  along  the  tra¬ 
jectory  since  the  rate  of  firing  of  the  side  thruster  is  not  always  a  fy>n«t»nt. 

In  order  to  compensate  exactly  for  gravity  there  must  be  a  predictable  amount 
of  impulse  imparted  to  the  missile  during  a  given  time  interval.  If  one  side  jet 
pair  fails  to  fire  at  the  proper  time  and  the  missile  ^develops  a  vertical  accelera¬ 
tion  due  to  gravity,  the  acceleration  must  be  corrected  and  the  missile  brought 
back  to  its  appropriate  flight  path  by  more  rapid  firing  of  the  side  thrusters. 

If  the  success-failure  sequence  is  known,  the  number  of  successful  side 
jet  firings  required  to  bring  the  missile  back  to  its  original  flight  path  can  be 
determined  by  flight  simulation.  For  the  simpler  cases  this  hps  been  done  and 
a  time  relation  between  state  position  and  the  required  number  of  successful 
firing  results  has  been  determined  explicitly.  The  more  complicated  sequences 
would  present  some  difficulty  especially  if  the  supply  of  available  side  jets  is 
exhausted.  On  the  average,  however,  a  missile  in  a  given  state  whose  vertical 
acceleration  has  been  arrested  will  have  experienced  a  predictable  number  of 
successes  within  a  given  time. 


QUESTIONS  TO  BE  CONSIDERED 

The  following  questions  are  submitted  for  consideration  by  the  panel. 

1)  The  distribution  of  the  mlasile  deviation  about  an  average  flight  path 
will  involve  the  dispersion  of  a  properly  functioning  missile  as  well 
as  the  dispersion  resulting  from  jet  failure.  A  technique  ia  required 
for  combining  the  two  sources  of  error. 

2)  Sinoe  the  target  is  at  a  random  position  relative  to  the  launcher,  the 
probability  of  hit  may  be  related  to  the  range  of  the  target.  How 
should  the  target  range  be  included  as  an  error  source? 

3)  Toward  the  end  of  flight,  the  number  of  jets  to  be  fired  bepomes 
limited.  Therefore,  if  a  failure  occurs  late  in  the  flight,  there  might 
be  empty  rows  of  Jeta.  This  would  result  in  a  delay  in  firing  after 
the  failure  until  a  pair  of  jets  would  be  in  the  proper  position  to  be 
fired,  How  should  this  problem  be  handled? 


A  PROBASILIW  APPROACH  TO  CATASTROPHIC  THREAT 


Clifford  J.  Maloney* 
Bethesda,  Maryland 


I.  INTRODUCTION.  Prior  to  World  War  II  statistical  methods  were 
viewed  as  a  body  of  techniques  appropriate  to  scientific  research  and 
to  a  limited  range  of  other  activities,  primarily  insurance.  The  war  ® 
and  the  immediate  post  war  years  saw  an  enormous  expansion  of  the  ap¬ 
plication  of  statistics  to  quality  control  of  incoming  supplies ,  opera¬ 
tions  research  as  a  replacement  for  trial  and  error  in  the  choice  of 
operational  organization  and  technique,  and  sample  surveys  as  a  substitute 
for  complete  canvass  for  obtaining  socio-economic  data  needed  in  decision¬ 
making.  Developments  more  purely  mathematical  in  nature,  Including  linear 
programming,  development  of  the  electronic  computer,  finite  mathematics 
and  the  like  served  to  help  foster  an  increasing  mathematical,  probabilisti 
and  statistical  approach  to  decision-making  in  all  aspects  of  human  activit 

Perhaps  the  one  aspect  which  has  so  far  not  attracted  the  attention 
of  scholars  has  been  the  anticipation,  prevention,  and/or  amelioration  of 
unexpected  but  costly  contingencies;  an  activity  the  businessman  has  long 
known  as  "putting  out  fires."  But  despite  the  fact  that,  superficially 
at  least,  "every  case  is  different,"  the  very  fact  that  in  the  aggregate 
these  occurrences  are  both  frequent  and  costly,  suggests  that  at  the 
very  least  we  can  attempt  to  codify  the  techniques  by  which  the  successful 
practitioners  cope  with  these  pheonomena ,  however  diversified,  and  that  a 
possibility  as  well  exists  that  some  underlying  structure  will  be  dis¬ 
covered. 


II.  THE  NATURE  OF  CATASTROPHE.  The  Statistical  Department  of  the 
Metropolitan  Life  Insurance  Company  defines  a  catastrophe  as  any  accident 
in  which  25  or  more  lives  are  lost.  Two  incidents  will. show  that  an 
event  may  be  a  disaster  despite  the  fact  that  no  loss  of  life  occurred 
or  was  even  threatened.  The  first  was  the  Pueblo  Incident  where  tne 
chief  cost  was  the  severe,  and  continuing,  but  unmeasurable  contribution 
to  the  threat  to  peace.  The  second  waa  the  public  reaction  to  the 
characterization  of  a  military  briefing  as  a  "brainwashing."  One  suspect*; 
that,  two  years  later,  the  reaction  would  be  very  different.  A  first 
attempt  to  list  the  characteristics  of  a  disaster  Is  given  in  Figure  1. 

The  ensuing  six  figures  give  a  few  ill-chosen  examples  under  each  of  six 
disaster  characteristics  to  which  the  reader  can  undoubtedly  add  an  in¬ 
definite  number  of  more  suitable  examples. 

My  claim  that  while,  in  fact,  unexpected  the  disastrous  event  was 
"predictable"  can  be  Illustrated  by  the  seizure  of  the  Pueblo.  All 
parties  recognized  that  such  an  act  was  conceivable.  The  difference 
lay  exclusively  In  their  assessments  of  its  likelihood.  All  1  mean  by 


♦Views  expressed  herein  are  those  of  the  author  and  not  to  be  construed 
as  official. 
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occurrence.  Contamination  of  the  moon  by  earth  microbes,  or  of  the 
earth  by  organisms  returning  with  the  astronauts  is  another  such  case. 

Earth  rupture,  tsunami,  or  earthquakes  are  all  conceivable  consequences 
of  underground  nuclear  tests.  It  is  unnecessary  to  argue  that  the  in¬ 
conceivable  never  happens  in  order  to  make  it  clear  that  most  catastrophes 
are  of  an  ordinary  garden  variety;  that  their  catastrophic  nature  lies  in 
their  (1)  severity  and  (2)  unexpectedness,  not  in  their  absolute  novelty. 

That  qjMostrophe  severity  is  as  much  psychological  as  real,  may  not 
be  at  once  apparent.  Such  loss  (other  than  one  ship  and  the  suffering 
of  the  men  and  their  families)  as  was  sustained  in  the  Pueblo  incident, 
lies  in  the  threat  to  world  peace  -  i.e. ,  is  psychological.  The  Dugway- 
sheep  kill  created  a  great  public  outcry  -  yet  the  anticipation  that 
several  workmen  will  lose  their  lives  when  the  Washington,  D.  C.  subway 
is  built,  won't  raise  an  eyebrow.  The  brainwashing  episode  was  an 
extreme  example. 

III.  DECISION-MAKING  DOMAINS.  To  decide  to  ignore  or  to  anticipate 
a  threat  Is  to  make  a  decision,  and  decision-making  ia  one  of  the  more 
recent  and  more  prestigious  applications  of  probability.  If,  as  claimed, 
catastrophic  threat  has  not  hitherto  been  rerognized  as  an  appropriate 
application  of  decision  theory,  there  must  be  something  about  the  material 
that  conceals  its  appropriateness  for  the  technique.  We  can  at  once 
expose  this  unconscious  assumption,  show  how  to  remove  it,  and  show  the 
absolute  necessity  of  doing  so  by  reviewing  the  circumstances  sin  which 
the  application  of  probability  is  currently  fully  recognized.  'First, 
partly  as  a  horrible  example,  partly  because  all  problems  not  treated 
with  a  formal  algorithm  are  necessarily  bo  treated,  and  partly  to  show 
the  absolutely  fundamental  role  of  probability  in  all  of  llfe'B  decisions, 
the  universally  applied  method  of  common  sense  will  be  listed  as  a  specific 
method  and  discussed.  Its  characteristics  are  given  in  Figure  8.  Many, 
if  not  most,  of  life's  decisions  hinge  on  an  estimate  of  probability.  To 
portray  this  rule  in  a  military  context,  I  have  extracted  certain  sen¬ 
tences  from  the  chapter  on  the  Anzio  Campaign  from  the  book  COMMAND 
DECISIONS,  published  by  the  Office  of  the  Chief  of  Military  History  in 
1959,  and  present  them  in  the  appendix.  Figure  9  is  a  map  of  the  theater 
involved.  By  reading  these  extracts,  which  are  limited  to  only  those 
sentences  in  which  a  probabilistic  concept  is  essential,  in  order  one  can 
gain  a  picture  of  the  campaign  almost  as  complete  as  by  reading  the  full 
text. 

Two  further  points  should  also  be  clear.  The  various  generals 
assessed  the  probabilities  very  differently.  In  particular,  General 
Mark  Clark  feared  over-precipitate  advance  because  of  his  one-time 
experience  at  Salerno,  The  English  authorities  seemed  to  feel  that 
the  German  Cassino  Line  would  crumble  the  moment  the  Allies  hsd  gained 
a  foothold.  No  Allied  commander  seems  to  have  contemplated  the  possibility 
of  an  unopposed  landing  -  which  in  the  event  is  what  happened.  What  I 


seek  to  infer  fro”  this  ■<«  that  up  to  the  time  of  World  War  11, 

no  formalized  procedure  existed  for  dealing  with  the  probability  com¬ 
ponent  of  comnand  decision-making  and  that  that  lack  proved  costly. 

The  application  of  probability  considerations  first  occurred,  and 
by  now  is  well  established,  in  the  situations  which  grew  out  of,  or 
which  can  be  logically  reduced  to,  games  of  chance.  In  practice,  these 
cases  arise  in  designed  experiments  or  in  sample  surveys  where  the  sample 
is  drawn  by  "probability  sampling."  This  situation  is  characterized  in 
Figure  10. 

In  a  closely  related  but  distinguishable  situation,  characterized 
in  Figure  11,  the  sample  is  not  drawn  from  a  preformed  population  but 
from  a  growing  one.  Both  of  these  cases  (Figures  10  and  11)  involve 
probabilities  and  sample  sizes  in  a  middle  range. 

In  the  next  situation  (Figure  12)  we  are  dealing  with  a  case  where, 
again,  probabilities  are,  or  at  lease  often  are,  in. the  middle  range, 
but  sample  sizes  are  usually  not  even  defined.  During  World  War  II,  the 
Weather  Bureau  developed  a  so-called  30-year  series  of  Northern  Hemisphere 
maps  to  enable  military  forecasters  to  make  forecasts  by  matching  the 
current  weather  map  with  an  earlier  situation,  on  the  assumption  that 
the  succeeding  weather  should  also  be  a  repetition.  Unfortunately, 
matching  is  never  exact.  At  this  point  a  new  factor  if  brought  into 
play,  which  goes  back  to  the  earliest  recognition  of  a  role  of  chance 
assessment  In  decision-making,  but  vhicVteas  relatively  neglected  in 
the  century  preceeding  World  War  II,  while  an  objective  foundation  for 
probability  seemed  to  be  proving  adequate.  This  new  factor  was  the 
concept  of  personal  probability,  not  as  a  quantity  having  external 
reality,  but  as  a  measure  of  subjective  mind  Btate  whether  shared  by 
several  minds  or  peculiar  to  one.  It  will  be  argued  below  that  this  > 
factor  has  been  Incorrectly  apprehended.  Here  it  is  sufficient  to  note 
that  on  this  foundation  a  super-structure  for  dealing  with  forecasting, 
technological  and  otherwise,  hasbeen  elaborated  and  is  being  increasingly 
widely  adopted.  A  good  Introduction  Is  afforded  by  Bright  (1968)  and 
the  references  there  included. 

Finally,  we  come  to  the  situation  contemplated  in  this  paper.  Like 
the  case  above,  sample  size  Is  essentially  non-existent.  But  here  the 
‘probabilities  are  so  low  that  no  hope  exists  of  accumulating  a  fund  of 
cases,  sufficiently  "similar"  to  provide  more  than  the  most  tenuous  basis 
for  probability  quantitation  even  as  a  belief  state.  This  is  the  case 
described  in  Figure  13.  The  essential  difference  between  this  case  and 
that  of  Figure  12  is  that,  whereas  in  the  latter,  probabilities  are 
fairly  high,  here  they  are  low.  For  example,  stock  market,  weather  or 
weapons  system  forecasting  cannot  successfully  be  based  on  simple  sample 
statistics,  as  for  instance  is  done  when  the  number  of  telephones  or 
children  per  household,  or  the  weight  gain  due  to  a  ration  additive  is 
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assayed  In  a  controlled  trial;  still  the  market  opens,  the  next  day 
cooes  and  brings  with  it  an  after  the  fact  verifiable  weather  outcome, 
and  whatever  weapon  system  is  chosen  for  development,  five  to  ten  years 
justifies  the  decision  or  reveals  its  Invalidity.  But  catastrophic  threat 
may  or  may  not  come  to  pass.  The  air  defense  of  the  United  States  against 
a  bomber  attack  by  the  Soviets  was  incredibly  expensive.  Was  it  a  success? 

No  such  attack  occurred,  but  would  one  have  had  one  if  the  defense  been  less  ef' 
fective  or  less  costly?  We  will  never  know.  The  current  debate  over  ABM 
deployment  would  not  continue  a  day  if  all  concerned  were  agreed  that 
within  the  decade  of  the  seventies  one  or  more  technologically  effective 
nuclear  missiles  were  highly  likely  to  be  launched  against  one  or  more  of 
our  cities.  The  ABM  proponents,  quite  as  much  as  the  opponents,  not  only 
hope  that  the  event  will  not  occur,  but  assess  its  likelihood  as  low.  The 
disagreement  is  entirely  over  how  low  is  low  enough. 

IV.  PROBABILITY  FOUNDATIONS.  Three  times  in  the  past  the  development 
of  a  new  foundation  for  quantitative  probability  has  resulted  In  a  major 
success  in  the  increase  in  understanding  and  control  of  the  real  world. 

The  two  "objective"  foundations ,  the  a  priori  or  necessary  of  Cardan 
Galileo,  Fermat  and  Pascal,  and  the  frequency  foundations  of  Graunt  and 
Halley,  developed  about  the  same  time,  proved  adequate  as  a  foundation  for 
all  applications  of  probability  prior  to  World  War  I  .  They  were  abstracted 
to  a  formal  mathematical  calculus  at  the  end  of  that  period  by  Kolaogoroff 
and  others. 

For  the  most  part  the  necessary  and  the  frequency  foundation  for 
probability  are  complementary.  There  is,  however,  one  aspect  in  which 
they  give  conflicting  counsel,  the  problem  of  outliers.  On  the  frequency 
definition  an  outlier  is  meaningless  -  for  that  approach  takes  experience 
as  given.  On  the  contrary,  the  a  priori  or  necessary  approach  rejects 
experience  whenever  it  is  in  conflict  with  doctrine.  Adherents  of  both 
approaches,  being  sensible  men,  adhere  neither  to  the  one  extreme  nor  to 
the  other.  Nor,  indeed,  do  they  in  practice  achieve  a  comfortable  com¬ 
promise. 

As  suggested  above,  notions  of  "degree  of  belief"  as  a  foundation  of 
probability  theory,  while  studies  in  the  years  following  World  War  I,  were 
not  widely  employed  until  after  World  War  II.  Most  adherents  of  personal 
probability  adopt  a  concensus  form  of  the  approach.  It  is  not  what  one 
believes,  but  what  one  ought  to  believe  that  matters.  DeFinetti  and 
Savage,  however,  go  all  the  way  and  allow  everyone  his  own  private  degree 
of  belief,  divorcing  the  concept  entirely  from  external  world  events. 

A  fifth  foundation  for  the  concept  of  probability  underlies,  if 
somewhat  subconsciously,  most  work  on  curve  fitting  and  indeed  comes 
close  to  formal  enunciation  especially  in  the  older  least  squares 
literature.  Here  every  value  of  every  quantitative  property  Is  assumed 
to  be  precisely  determined,  but  as  a  function  of  a  literally  infinite 
number  of  independent  or  predictor  variables,  say 
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X-a+B+o  +  6+  c+  .  .  .  . 


One  achieves  a  chance  structure  by  neglecting  most  of  these  latter.  Then 
the  precise  value  of  the  predicted  variable  depends  on  the  contribution 
of  the  neglected  variables,  which  being  unknown  makes  the  predicted  value 
a  chance  one.  Each  of  the  other  theories  can  be  obtained  by  making  assump¬ 
tions  about  this  formula.  The  a  priori  definition  assumes  that  the  relative 
frequency  of  possible  values  of  X  can  be  Inferred  because  the  values  of 
the  independent  variables  are  known,  whenever  individually  important.  The 
frequency  definition  conversely  finds  the  values  only  of  a  fraction  of  the 
parameters,  age,  sex,  occupation  and  so  on  in  the  case  of  mortality  schedules 
ascertainable,  but  a  number  of  essential  parameters  remain  unknown,  so  that 
observations  on  X,  as  well  os  calculations  on  the  known  right  hand  terms, 
are  required.  Either  subjective  theory  would  differ  from  the  a  priori 
approach  only  in  that  the  values  of  the  parameters  and  the  form  of  the 
function  would  be  apprehended  intuitively  rather  than  be  calculated  from 
theory  or  inferred  from  sampling  experiments.  How  does  a  batter  hit  a 
ball  is  the  classic  example  to  distinguish  between  the  rational,  algorithmic 
approach  to  nature  and  the  intuitive  subconscious  unanalyzed  approach.  The 
batter  seldom  regards  a  graduate  degree  in  mechanics  as  essential  for  suc¬ 
cess  -  though  he  does  Indeed  bear  In  mind  all  the  identifiable  admonitions 
which  can  be  derived  either  from  theory  or  experience. 

Personal  probability  is  to  be  distinguished  from  objective  probability 
not  because  It  is  an  entirely  different  -  an  unrelated  -  species,  but  be¬ 
cause  it  stresses  a  component  always  present  when  a  mind  contemplates  the 
external  world.  The  mind  never  knows  the  world,  It  has  only  beliefs  about 
it.  These  beliefs  are  a  substitute,  or  better,  an  alternative  for  objec¬ 
tive  measures.  In  some  contexts,  the  objective  criteria  prove  more  suc¬ 
cessful,  in  others  the  subjective  judgments.  These  beliefs  can  have  little 
value  if  not  verified  in  practice,  and  verified  in  just  the  way  that  obj¬ 
ective  deductions  are  -  by  the  observed  outcome.  If  the  odds  makers  pick 
the  wrong  team  to  win  the  World's  Series,  all  agree  that  the  choice  wee  an 
error.  One  is  entitled  to  his  opinion  -  but  only  before  the  game.  The 
several  bases  for  the  concept  of  probability  are  set  out  in  Figure  14. 

V.  A  NEW  VIEW  OF  PROBABILITY.  It  was  remarked  above  that  the 
requirements  of  supplying  a  foundation  for  decision  theory  in  circum¬ 
stances  where  evaluating  parameters  by  sampling  of  many  indistinguishable 
elements  is  not  available,  has  since  World  War  II  been  furnished  by  the 
concept  of  probability  as  "degree  of  belief."  It  was  further  asserted 
that  the  true  nature  of  personal  probability  is  not,  as  often  thought, 
something  apart  from  objective  probability.  There  is  only  one  probability. 
That  is  the  success  ratio  in  a  series  of  trials.  A  logical  complication 
arises  from  the  fact  that  the  series  is  necessarily  infinite  and  there¬ 
fore  necessarily  conceptual  rather  than  actual.  It  may  be  estimated  in 
each  of  three  ways.  The  first  way  is  the  a  priori  or  necessary  approach. 
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If  the  possible  results  can  be  enumerated,  if  they  possess  as  adequate 
degree  of  s yam e try ,  and  if  they  are  independent,  we  apply  principle 
of  insufficient  reason.  This  approach  haa  always  been  intuitively  ap- 
pealing,  logically  Intractable,  and  limited  in  application.  The  device 
of  just  trying  or  observing  a  large  number  of  trials,  partially  over¬ 
comes  this  latter  difficulty,  shares  in  a  different  way  the  intuitive 
appeal,  but  introduces  its  own  logical  difficulties.  The  degree  of 
belief  approach  possesses  the  prime  advantage  of  always  being  applicable, 
but  only  to  the  perceiving  mind,  never  to  the  real  world.  For  this  latter 
to  occur  we  must  appeal  to  a  method  of  verification.  For  example,  consider 
a  weather  forecaster.  Each  newscast  he  preaenta  a  "fearless  forecast" 
preceded  by  an  explanation  of  why  his  last  forecast  failed.  Heather 
forecasts  are  made  on  the  basis  of  much  data,  much  theory,  historical 
records,  or  Farmer's  Almanacs.  The  basis  is  irrelevant.  The  verifica¬ 
tion  is  the  thing.  If  the  forecasts  are  verified,  the  forecaster  is  a 
success.  The  sequence  of  trials  is  thus  the  successive  forecasts  periods. 
The  successes  are  the  verified  forecasts,  all  others  are  failures.  The 
process  is  just  as  objective  as  was  Graunt's  birth  series.  That  the  fore¬ 
caster  used  his  degrae  of  belief  in  formulating  his  forecast  is  as  inci¬ 
dental  as  the  use  by  a  batter  of  hlB  in  deciding  to  swing  at  a  pitched 
ball.  Each  man  functions  as  a  measuring  Instrument.  A  person  is  often 
superior  to  s  machine  in  such  functions. 

The  catastrophic  threat  problem  does  Introduce  a  complication.  In 
most,  if  not  all,  realistic  applications  of  personal  probability  tha 
verification  is  not  long  in  coming.  That  Is  why  people  forecast.  But 
a  catastrophic  threat  la  necessarily  a  rara  evant,  If  It  happens  at  all. 
Just  as  we  cannot,  before  tha  event,  accumulate  an  adequate  fund  of 
experience  upon  which  to  base  en  estimate  of  probability,  so  we  cannot 
after  the  Verification  step  accumulate  a  sufficient  fund  of  verified  or 
disproved  forecasts  to  establish  a  success  ratio  in  a  trial  sequence 
known  to  be  homogeneous  in  the  probability  sense.  Fortunately,  the 
occurrence  compensates  for  its  severity  by  its  rarity.  The  new  element 
which  the  present  analysis  seeks  to  supply  to  the  contribution  of 
probability  to  decision-making  is  s  new  technique  of  verification. 

For  this  purpose  consider  Figure  15.  Here  we  have  listed  not  one 
stochastic  sequence,  but  three  labelled  as  conditions  A,  B,  C.  A,  B, 
and  C  could  be  threats  of  aggression  by  three  different  nations,  or  the 
threat  of  war,  of  famine  and  of  a  major  epidemic.  Each  represents  what 
would  usually  be  chosen  to  characterize  one  aeries  of  trials  as  ordinarily 
described  in  discussions  of  probability.  But  the  types  of  catastrophic 
threat  are  not  limited;  they  are  legion.  Suppose  we  employ  the  methods 
of  personal  probability  to  assign  "degrees  of  belief"  to  occurrence  of 
each.  Further,  let  this  be  done  not  by  one  individual,  or  by  one 
forecasting  team,  but  by  cany,  each  applying  whatever  "betting  system" 
is  most  attractive.  We  would  get  as  a  result  a  two-fold  matrix  of 
assigned  probabilities  such  as  is  diagrammed  in  Figure  16. 
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While  it  is  true  that  the  probabilities  subjectively  assigned  to 
the  contingency  that  a  catastrophe  of  specified  nature  would  occur  in 
a  specific  interval  of  time  is  low,  and  in  general  varies  as  between 
the  types  of  contingencies  involved,  the  possible  contingencies  are 
legion,  so  that  it  is  to  be  expected  that  on  any  given  estimation 
procedure  a  fairly  high  number  of  possible  catastrophes  would  be 
assigned  the  same  or  nearly  the  same  value  of  the  probability  of 
occurrence  In  a  specific  time  frame.  In  Figure  17,  this  is  indicated 
by  assigning  probabilities  not  to  individual  catastrophic  contingencies, 
but  to  classes  characterized  by  the  property  that  within  a  class,  each 
individual  catastrophe  is  regarded  as  equally  likely.  The  grouping  may 
well  be  different  as  between  rows.  Hence,  the  columns  do  not  refer  to 
the  same  contingencies  in  every  row.  The  letter  k  in  each  cell  represents 
the  number  of  contingencies  all  assigned  the  same  probability  of  occur¬ 
rence  by  the  technique  of  that  given  row. 

As  each  period  of  observation  passes,  the  occurrence  of  the  various 
types  of  catastrophe  would  be  noted.  The  subjective  probabilities  as¬ 
signed  by  any  one  procedure  (in  any  one  row)  would  be  verified  or  refuted 
according  as  the  empirical  success  ratio  was  sufficiently  close  to  the 
ex  ante  assigned  probability  or  not.  This  empirical  success  ratio  would 
be  calculated  in  a  elightly  different  manner  from  that  utilized  at  pre¬ 
sent,  where  it  is  assumed  that  every  sequence  of  trials  is  studied  in 
isolation  and  without  regard  to  what  is  happening  in  any  other.  Let  the 
success  ratio  at  a  certain  observation  period  be  n/m;  where  n  is  the 
number  of  catastrophes,  all  having  been  assigned  the  same  probability, 
which  have  so  far  occurred  and  m  Is  the  product  of  the  number  of  such 
catastrophes  by  the  number  of  observation  periods.  Here  k  equals  the 
number  of  catastrophes  grouped  as  having  the  same  subjective  probability 
of  occurring,  and  £  la  the  probability  Itself.  Let  h^ be  the  number  of 
catastrophes  of  this  class  which  occur  in  the  next  observation  period. 

Of  course,  h  will  almost  always  be  zero,1 rarely  be  unity  and  almost 
never  be  greater.  Then  the  success  ratio  at  the  end  of  this  period 
becomes 


n  4-  h 
m  +  k 

The  essence  of  this  procedure  is  that  we  are  judging,  not  the  auccess 
in  assigning  subjective  probabilities  to  specific  types  of  catastrophes, 
but  the  betting  system  Itself  (at  this  level  of  probability).  If  the 
probability  assignment  for  one  class  of  catastrophe  is  verified,  our 
confidence  in  the  assignment  of  all  classes  is  strengthened. 

So  far  we  have  pooled  experience  in  assessing  the  efficacy  of  a 
particular  "betting  system,"  i.e.,  method  of  assigning  subjective 
probabilities,  but  only  at  a  specific  level  (or  limited  range  of  levels) 
of  the  assigned  probability.  As  first  shown  by  Karl  Pearson,  we  can  use 
chi-square  or  some  alternative  approach  to  get  a  combined  test  of  the 
procedure  irrespective  of  level. 
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Particularly  in  a  medical  environment  Luc  need  to  insure  a  very 
high  level  of  safety  for  products  produced  for  human  consumption  has 
received  increasing  attention.  The  Polio  and  Thalidomide  incidents 
are  instances.  The  cigarette  and  cyclamate  episodes  provide  an  il¬ 
lustration  of  variation  in  response  to  evidence.  These  cases  have 
been  studied  (so  far  as  the  author  la  aware)  by  extrapolation  of  con¬ 
ventional  statistical  techniques  from  regions  of  relatively  high 
probability  (so  thAt  evidence  is  attainable)  to  the  region  of  Interest 
where  probability  of  occurrence  la  extremely  low  -  one  in  a  million 
exposures  or  leas.  The  present  paper  seeks  to  provide  a  procedure 
for  catastrophes  where  even  this  extrapolatory  technique  is  unavailing, 
but  there  is  no  reason  why,  when  it  is,  that  the  evidence  from  both 
approaches  should  not  be  pooled. 

VI.  THE  COST  FACTOR.  Costs,  like  threats,  are  some  real  and  some 
imaginary.  A  perfect  system  of  probability  assessments  would  still  not 
be  adequate  for  decision-making,  if  costs  are  ignored,  except  in  those 
instances  where  the  probability  concerned  Is  negligible. 

There  is  a  vague  appreciation  that  in  a  military  context  "negligible" 
probabilities  are  a  trap.  Defenders,  particularly  budget  officers  who 
are  to  fund  protection  from  such  "impossible"  threats,  or  competitors 
for  those  funds  inevitably  dlsmiea  such  probabilities  as  negligible, 
yet  often  in  fact,  just  that  stratagem  will  be  selected  by  attackera 
in  the  very  knowledge  that  it  will  be  unanticipated  by  the  defenders. 
Instances  arc  the  Pearl  Harbor  attack;  th*  choice  of  land  ovar  sea 
approach  at  Singapore;  the  Black  Forest  end  run  of  the  Maginot  Line; 
the  "post  season"  sea  assault  at  the  Rattle  of  Hastings.  Two  famous 
examples  from  the  history  of  mathematics  describe  the  failure  of 
Saccherl,  and  the  success  of  Hamilton  from  a  confrontation  with  the 
unthinkable. 

As  this  paper  treats  catastrophic  threat  the  coats  are  by  assumption 
high.  But  a  closer  estimate  is  needed.  Perhaps  the  outstanding  charac¬ 
teristic  of  disaster  coBts  is  that  the  ex  post*  estimates  of  costs 
"always"  outweigh  the  ex  ante^  cost  estimates,  and  by  a  huge  factor. 

This  is  recognized  by  every  parent  who  tries  to  get  his  teenage  off¬ 
spring  to  cross  a  college  campus  or  to  drive  the  family  car  with  due 
caution.  The  military  meets  this  problem  in  stark  terms  when  it  tries 
to  indoctrinate  habits  of  safety  in  new  recruits.  Live  ammunition  in 
field  exercises  represents  one  attempt  to  secure  credible  reality.  How¬ 
ever,  no  adequate  technique  has  been  found. 


'before  the  observation  period, 
after  the  observation  period. 
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But  while  a  closer  approximation  of  ex  ante  to  ex  post  costs 
assessments  by  new  troops  would  reduce  the  incidence  of  nemnn/il 
tragedy,  and  shorten  the  battle  seasoning  of  troops,  the  wide  dis¬ 
crepancy  inevitably  existing  in  assessing  the  costs  of  catastrophes, 
is  the  essence  of  the  catastrophic  threat  problem.  Indeed,  in  practice 
all  or  nearly  all  the  discussion  will  be  found  to  turn  on  an  estimation 
of  the  probabilities,  and  little  of  it  on  assessment  of  the  costs,  though 
a  gti.it  deal  of  attention  iis  devoted  to  deprecating  the  costs  of  preven¬ 
tion  or  of  protection  predicated  on  this  presumed  negligible  risk.  With 
a  one-sldt d  consideration  of  both  costs  and  probabilities,  there  is 
little  chance  of  efficient  decision-making. 

The  instint tive  impulse  of  the  mathematically,  or  philosophically 
minded  is  to  generalize  the  problem  and  then  return  to  the  specific 
example  -  In  our  case  the  unreality  of  ex  ante  estimates  -  on  the  basis 
of  supposedly  clearer  insight.  To  estimate  is  to  make  a  decision,  and 
decision-making  Is  intrinsically  emotional.  This  point  was  graphically 
demonstrated  by  Koehler  in  experiments  during  World  War  1,  Figure  18. 

If  a  chicken  is  placed  close  to  a  short  stretch  of  fence  and  a 
handful  of  grain  ie  placed  on  the  other  side  close  to  the  fence,  but 
beyond  reach  through  It,  the  chicken  will  excitedly  press  against  the 
fence  and  never  step  beck  to  seek  a  way  around  it.  If  the  grain  is 
moved  farther  awav.  the  emotional  attraction  will  be  reduced,  and  the 
hen  will  have  a  better  chance  of  solving  the  problem. 

So  true  is  it  that  dacision-maklng  is  an  emotional  process  that 
advertising  and  political  discussion  are  directed  almost  exclusively 
at  emotional,  and  only  incidentally  at  logical  components  of  issues. 

In  consequence  persons  who  will  make  decisions  ere  selected  on  the 
basis  of  emotional,  not  rational,  characteristics.  So  widespread  is 
this  process  that  it  almost  becomes  a  "first  law  of  management"  that, 
when  it  comes  to  making  decisions  "those  with  the  power  leek  the  know¬ 
ledge,  those  with  the  knowledge  lack  the  power."  In  consequence,  the 
"on  tap"  become  the  "untapped." 

This  lack  of  knowledge  has  not  merely  been  recognized,  but  has 
been  deplored  through  all  of  history.  Even  so,  courts  of  inquiry, 
inquests,  grand  juries,  staffs,  study  groups  end  a  host  of  other 
techniques,  have  evolved  to  supply  knowledge  deficiencies  of  those  in 
power.  However,  in  a  power  rather  than  information  dominated  environ¬ 
ment,  all  such  devices  tend  to  be  ineffective  because  they  tend  to  be 
ignored.  Where  power  makes  decisions,  only  power  can  influence  decisions 
Where  decisions  are  out  of  touch  with  reality  because  information  is 
ignored,  and  countervailing  power  is  laking  feedback  from  consequences 
is  avoided.  Figure  19  lists  certain  principles  which  seem  applicable. 

Ascendency  of  emotional  over  informational  factors  in  decision¬ 
making  leads  to  a  widely  recognized,  widely  deplored,  oscillation  in 
efforts  at  averting  or  dealing  with  catastrophic  threat.  The  soldier 
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knows  this  oscillation  in  the  contrast  between  his  hero's  role  in 
times  of  threat,  and  'a  role  in  seeming  peace.  7I»*  tin- roof 

parable  is  known  to  all,  but  heeded  by  few.  The  technique  outlined 
in  this  paper  represents  only  what  could  be  done,  but  not  what  will 
be  done,  until  the  forces  producing  this  violent  oscillation  are 
rendered  impotent.  A  single  incident  at  this  Design  Conference  will 
Illustrate  the  situation.  One  talk  at  the  Conference  was  given  by 
Dr.  Condon,  in  charge  of  the  safety  program  for  the  moon  landings. 
After  explaining  that  the  success  of  the  safety  program  was  due. 

In  large  part  at  least,  to  his  refusal  to  adopt  a  "good  enough" 
attitude,  the  speaker  conceded  that  the  program  was  thereby  made 
costly  and  commented  that,  in  the  days  ahead  there  will  be  "pressures 
on  us  in  the  way  of  costa." 

This  Is  the  universal  story  of  the  fight  against  catastrophe. 

The  expense  ensures  success;  the  success  breeds  lack  of  support. 

It  will  always  be  thus  where  decision-making  is  a  budgetary  process. 
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RARE 

Figure  1.  Characteristics  of  Catastrophes 


SUDDEN 

Donora  vs.  Los  Angeles  Smog 
Iran  Earthquake  vs.  Erosion 
Epidemics  vs.  Normal  Death  Rate 
Air  Crash  vs.  Automobile  Deaths 
Poliomyelitis  vs.  Cardiac  Arrest 

Figure  2.  illustrative  Examples 
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1942 

Boston 

Club  Fire 

492 

1944 

Port  Chicago 

Ammo  Ships 

322 

1944 

Hartford 

Circus  Fire 

168 

1947 

Southwest 

Tornado 

167 

Figure  3.  Illustrative  examples 


UNEXPECTED 


Thalidorride  Teratogency 
Pearl  Harbor  Attack 
Prince  of  Wales  and  Repulse 
Virginia  Flood  (1969) 

Figure  4.  Illustrative  examples 


PREDICTABLE 

Radium  illuminated  dials 
Giant  Solar  Flare 
.  Unknown  Moon  Agent 
Smallpox  among  natives 
Nuclear  test  triggered  tsunami 
Figure  5.  Illustrative  exanples 

PSYCHOLOGICAL 
Battle  of  Big  Bethel 
Sheep  deaths  near  Eugway 
D.  C.  Subway  deaths 
"Brainwash"  political  death 
Figure  6.  Illustrative  examples 
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Figure  7,  Putting  out  fires 
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Figure  Coicon  sense 


Preceding  page  blank 


RANGE 


aiddle 


TrntMtrrum 
i  L.UU1  IVyULi  ~  — 

-  -  -  formal  staciscics 

SAMPLE  SIZE  - 

-  -  -  large 

VALIDATION-  - 

-  -  -  experiment 

Figure  10. 

Formal  statistics 

RANGE  - 

-  middle 

TECHNIQUE  -  - 

-  -  -  quality  control 

SAMPLE  SIZE  - 

-  -  -  large 

VALIDATION - 

-  -  -  feed  back. 

Figure  11. 

Process  monitoring 

RANGE  -  -  -  - 

-  -  middle 

TECHNIQUE  -  - 

-  -  -  forecast 
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-  -  -  feed  back 

Figure  12. 

Technological  forecasting 
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Figure  1J. 

Catastrophic  threat 
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PROBAB ILITY  FOUNDAT IONS 


Necessary  Fermat,  Pascal 

Frequency  Graunt,  von  Mises 

Abstract  Kolmogoroff 

Personal  Bernoulli,  Bayes 

Neglected  Causes 

Figure  14. 
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Figure  IS.  Examples  of  chance  sequences 
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Figure  16.  Matrix  of  probabilities  assigned  to  catastrophes 
(columns)  by  different  procedures  (rows) 


567 


11* 

P11 

K12’  P12 

....  Klc, 

Plc 

21* 
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K  , 
rc 

Prc 

Figure  17.  Matrix  of  catastrophes  assigned  a  common  probability 


KOEHLER  EFFECT 


Figure  18.  Effect  of  emotional  intensity  on  decision-msMng 
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Feedback  necessarily  occurs 
Information  feedback  is  cheapest 
Information  feedback  is  lgnorable 
Feedback  through  channels  is  no  feedback 

Figure  19.  Principles  of  decision-making  verification 
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APPENDIX 


1.  A  conpc.uJer  can  make  a  decision  simply  by  ruling  out  what  appears 
to  bln  to  be  Impractical  or  unfeasible.  (244) 

2.  General  Alexander  felt  that  ...  allied  troops  on  the  enemy  flunk" 
below  Rome  might  so  threaten  German  communications  as  to  compel  the 
enemy  to  retreat.  (248) 

3.  Link-up  between  Che  main  and  the  Anzlo  fronts,  It  was  assumed, 
would  take  place  no  later  than  seven  days  after  the  landing.  (248) 

4.  A  strengthened  Anzlo  force,  If  assured  continuous  resupply  by 
water,  could,  he  believed,  consolidate  a  beachhead....  (249) 

i 

5.  Whether  the  60  miles  between  Anzlo  and  the  Garlgliano  was  too  grunt 

a  distance  for  action  on  one  front  to  influence  the  other  was  alscuS'ied, 
but  it  was  accepted  as  an  unavoidable  risk.  (249)  | 

6.  It  was,  impossible  to  predict  the  exact  German  reaction  to  a  landing, 
but  the  most  probable  reactions  seemed  desirable  from  the  Allied 
standpoint.  (250) 

7.  The  Anzlo  force  might  provoke  the  Germane  ...  to  withdraw.  (250) 

8.  ...  intelligence  officers  of  the  15th  Army  Group  were  rather 

optimistic.  (251)  i 

9.  ...  they  'counted  on'  the  effect  of  weather  and  on  harassment  by 
the  Allied  air  forces  to  interfere....  (251) 

10.  The  [ambiguity  of  orders  to  Sixth  Corps]  arose  from  the  difficulty 
of  judging  ....  (252) 

11.  Fifth  Army  intelligence  estimates  were  less  optimistic....  (252) 

12.  The  enemy  was  judged  to  have.  (252) 

13.  By  the  third  day  the  Germans  could  perhaps.  (252) 

14.  Two  additional  divisions  could  probably.  (252) 

15.  The  Fifth  Army  assumed  that  the  VI  Corps  would  meet  strong  r i uUi* 
on  the  beaches  (252) 

16.  It  expected  the  Corps  to  receive  heavy  Counterattacks .  . .  (75') 

17.  ...  having  underestimated  German  strength  et  Salerno....  (252) 

18.  The  Fifth  Army  -  and  with  it  the  VI  Corps  -  expected  the  sau.v 
pattern. . . .  (252) 
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19.  The  Fifth  Army  expected  the  VI  Corps  to  be  ready  to  do  one  of  two 
things  upon  landing.  (253) 

20.  The  operation  becomes  such  a  desperate  undertaking.  (254) 

21.  Otherwise  "a  crack  on  the  chin  is  certain"  (254) 

22.  A  failure  now  would  ruin  Clark,  probably  kill  me,  and  certainly 
prolong  the  war...  (254) 

23.  A  week  of  fine  weather  at  the  proper  time  and  1  (Lucas)  will 
make  it ...  .  (255) 

24.  Alexander  told  Lucas  "we  have  every  Confidence  in  you"  (255) 

25.  What  troubled  General  Lucas... was  the  contrast  between  his  own 
concern ...  and  nonchalance  in  the  higher  echelons....  (255) 

26.  Lucas  was  not  so  sure.  (255) 

27.  The  chances  are  seventy  to  thirty  that  (256) 

28.  He  [Lucas ]  believed  his  forces  lacked  the  strength  (256) 

29.  The  general  idea  Beams  to  be...  (256) 

30.  I  wish  the  higher  headquarters  were  not  so  optimistic....  (25b) 

31.  Securing  a  beachhead  was  all  Fifth  Army  expected....  (257) 

32.  Lucas  [was  not]  to  push  on  to  the  Alban  Hill  mass  at  the  risk  of 
sacrificing  his  corps  (257) 

33.  Such  a  possibility  [moving  on  the  Alban  Hills]  appeared  slim  to 
the  Fifth  Army  Staff  (257) 

34.  The  staff  questioned  Lucas'  ability....  (257) 

35.  It  was  obvious  what  the  loss  of  the  supply  base  would  mean  (257) 

36.  If  the  enemy  came  to  Anzio  in  strength  (257) 

37.  The  British  feared  they  might  mistake  Americans  for  Germans  (257) 

38.  What  everyone  had  overlooked  ...  was  the  possibility  of  achieving 
complete  surprise  (258) 

39.  The  Germans  always  regarded  the  long  sea  flanks  in  Italy  as 

exposed ....  (258) 
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40.  To  reinforce  (local  troops  Kesselring]  expected  to  call  on  Tenth 
Army  for  a  division....  (259) 

41.  He  hoped  to  have  the  Fourteenth  Army  in  north  Italy  move  ...  the 
equivalent  of  about  one  or  two  divisions  (259) 

42.  Fearing  that  the  Fifth  Army  was  about  to  make  a  breakthrough....  (259) 

43.  Feeling  that  the  fate  of  the  Tenth  Army  ....  (259) 

44.  According  to  the  German  estimate  the  landing  had  a  good  chance...  (259) 

45.  Field  Marshall  Kesselring  assumed  that  the  troops  would  probably  try. 
to  seize  the  Alban  Hills  (260) 

46.  The  Germans  were  Considerably  reassured  by  Allied  behavior  at  the 
landing  (261) 

47.  Keasalring'a  order  to  stand  fast  on  the  Garlgliano-Rapldo  line  was... 
in  the  nature  of  a  gamble....  (261) 

48.  If  the  Allies  attacked  on  January  23  or  24,  German  forces  would  not 
be  strong  enough  to  hold  (261) 

49.  The  evening  of  23  January,  Kesselring  "believed"  that  the  danger  of 
a  beachhead  expansion  was  no  longer  imminent  (261) 

50.  By  24  January  the  German  command  considered  the  danger  of  an 
Allied  breakthrough  removed.  (261) 

51.  Alexander  was  very  optimistic,  Clark  somewhat  subdued  (261) 

52.  Lucas'  concern  with  logistical  aspects  came  not  only  from  prudence 
(262) 

53.  He  believed  the  Germans  could  increase  their  build-up  (262) 

54.  He  believed  the  Germans  would  stop  his  VI  Corps  befors  it  could  cut 
their  line  of  communication  (262) 

55.  His  intelligence  officers  informed  him  that  the  Germans  were  taking 
troops  from  the  Fifth  Army  main  front  to  oppose  him  (262) 

56.  This  might  permit  the  Fifth  Army  to  advance  (262) 

57.  The  Fifth  Army,  Lucas  was  certain,  would  atill  have  to  fight  povarful 
rear  Ruards  (262) 

58.  He  expected  no  spectacular  rapidity  of  movement  (262) 
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59.  ...  he  sought  to  build  up  his  strength  and  his  supplies  to  remain 
intact  even  though  isolated  (2bi) 

60.  I  feel  now  [January  25]  the  beachhead  is  safe  (263) 

61.  Lucas  expected  the  1st  Armored  Division  to  arrive  soon  (263) 

62.  That  is  about  all  1  can  supply  but  1  think  it  will  be  enough  (263) 

63.  I  must  do  nothing  foolish  (263) 

64.  I  must  hold  it  I  think  I  can  (263) 

! 

65.  Kesselring  came  to  the  conclusion  that  the  Allies  were  preparing  a 
full  scale  attack  (263) 

66.  The  best  defense,  he  felt,  was  an  attack  on  hiB  own  (263) 

>  I 

67.  Lucas  thought  ha  could  attack  In  a  few  days  (264) 

i 

/ 

68.  He  expected  30  LST's  to  ba  unloaded  at  Anzio  27  January  (264) 

i 

69.  Clark  "received  the  Impression"  that  the  outcome  of  the  struggle 
depended  on  who  fcould  increase  hie  forces  more  quickly  (264) 

i 

70.  Though  the  'situation  was  not  clear  to  Clark  (264) 

72.  Apparently,  some  of  the  higher  levels  think  1  have  not  advanced  with 
maximum  speed  (264) 

73.  I  think  more  has  been  accomplished  than  anyone  had  a  right  to 
expect.  (264) 

71.  He  urged  Lucas  to  take  bold  offensive  action  (264) 

74.  Thla  venture  was  always  a  desperate  one  (264) 

75.  I  could  never  see  much  chance  for  it  to  succeed  (264) 

76.  Without  Anzio  our  situation  would  have  been  deaperate  (264) 

77.  Had  1  rushed  troops  to  Albano  and  Velletrl  they  would  have  been 
destroyed  (264) 

78.  The  only  thing  to  do  was  what  I  did  (264) 

79.  Kaep  the  enemy  off  balance  until  the  Corp  was  ashore  end  everything 
was  set  (264) 
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M.-  situation  is  crowded  with  doubt  and  uncertainty  (265) 

:■  expect  to  be  counterattacked  in  the  morning.  (265) 

\ 

i  think'\he  realized  the  aerlouaneaa  (265) 

He  [Clark]  thinks  I  should  have  bean  more  agressive  on  D-Day  (265) 

T:u.  iv  has  been  no  chance  to  build  "Shingle"  up  to  decisive  strength  (265) 
Anyone  could  have  seen, that  from  the  start  (265) 

•  an  win  if  1  in  left  alolv^  (265) 

:  ol.h’i  know  whether  1  can  stand  the  strain  (265) 

x_ 

'ark  and  those  above  him  thought  Ansio  would  shake  the  Casaino 

;  ,  :.i:  it  once  (266) 

no'/  had  no  right  to  think  that  (266) 

It  was  clear  that  the  attack  had  not  accomplished  much  (266) 
i'il  Sanger  force  met  unanticipated  opposition  (266) 

i hi*  enemy  had  an  unexpectedly  strong  and  well  organized  defensive 

’  •  U<;6) 

•  co mu 1 1  ulnar  the  Germans  has  built  up  their  forces  around  Anzio  (266) 

nit  ilio  Allies  did  not  know  was  how  doss  they  came  to  breaking  out 

•  lino  ad  (266) 

•  ■  Ail  lei  Intelligence  officers  teemed  like  overwhelming  German 

intelligence  officers  had  to  assume  (266) 

1  ;  .i-  ought  he  could  support  two  more  divisions  at  Anzio  on 


jo-.!  Clark  decided  that  the  enemy  build-up  dictated  a 
■  •■lenhivo  tactics  (267) 

t  February  "keeping  the  enemy  off  balance"  was  a  forlorn  hope  (267) 
no  disappointed  but  there  is  no  military  reason  why  they 
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102.  General  Devers  thought  Lucas  should  have  gone  on  -  on  landing  (268) 

103.  "Had  1  done  so,  I  would  have  lost  my  corps"  (268) 

104.  Clark  thought  Lucas  had  done  all  ha  could  at  Anzlo  (268) 

105.  I  thought  I  was  winning  something  of  a  victory  (268) 

106.  General  Clark  thought  Lucas  could  have  taken  the  Alban  Hills  but 
could  not  have  held  them  (268) 

107.  Clark  thought  British  G-2  intelligence  was  always  over  optimistic  (269) 

108.  The  Germans  built  up  their  differences  at  Anzlo  much  faster  than 
the  British  believed  possible  (269) 

109.  Clark  had  always  felt  that  Anzlo  had  little  chance  of  success  (269) 

110.  In  retrospect  Clark  felt  that  the  total  losses  at  Garlgliano  and  at 
Anzlo  might  have  been  safer  and  as  productive  at  Garlgliano  alone  (269) 

111.  A  powerful  counter  attack  at  Anzlo  could  well  have  wrecked  the 
entire  Italian  Campaign  (269) 

112.  By  the  and  of  January,  Clark  was  disappointed  by  Lucas'  lack  of 
aggressiveness  (269) 

113.  Clark  believed  Lucas  should  have  made  a  reconnaissance  in  force 
to  capture  Cisterns  and  Campoleone . . , .  (269) 

114.  Clark  thought  such  an  effort  to  be  not  incommensurate  with  Lucas' 
forces  (269) 

115.  Others  felt  muth  the  same  (269) 

116.  General  Marshall  thought  Lucas  could  have  taken  the  Albdn  Hills  (269) 

117.  However,  he  thought  Lucas  had  acted  wisely  (269) 

118.  Marshall  felt:  Lucas  could  not  have  held  tha  Alban  Hills  and  the 
port  at  Anzlo  (269) 

119.  The  theater  G-2  had  held  the  same  opinion  at  the  Christmas  Day 
Conference  at  Tunis  (269) 

120.  G-2  thought  Lucas  would  have  been  in  a  bad  way  without  a  main  front 
breakthrough.  (269) 
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121.  The  Allies  would  be  unable  to  keep  the  Germans  from  shifting  forces 
to  Anzlo  from  south  Italy  as  well  as  elsewhere  (269) 

122.  General  Lcmnitzer  also  felt  the  Allies  did  not  have  the  strength  to 
hold  the  Alban  Hills  (269) 

123.  Lemnitzer  thought  tnat  Alexander  hoped  that  the  Anslo  operation 
plus  a  main  force  attack  "might"  force  a  German  withdrawal  (270) 

124.  The  advance  "on"  the  hills  was  exactly  what  Alexander  thought 
possible  (270) 

125.  When  Alexander  visited  tee  beachhead  on  D-Day  he  approved  the 
decision  not  to  push  out  far  from  Anzlo.  (270) 

126.  Lemnitzer  thougnt  that  Alexander  thought  that  Lucas  had  done  no 
wrong,  but  was  under  too  much  strain  (270) 

/ 

127.  By  that  time  it  was  clear  the  Anzlo  operation  would  Involve  a 

long,  hard  struggle  (270)  /  / 

128.  It  would  seem  that  Lucas'  action  during  the  first  few  days  was 

justified  (270)  /  | 

a  / 

129.  The  main  German  Army  showed  no  signs  of  withdrawing  (270) 

'  f  * 

130.  The  Allies  saw  ho  Immediate  prospect  of  forcing  a  general  retreat  (270) 

,  / 

131 .  It  became  far  more  likely  the  Germans  would  move  in  strength  against 

Anzio  (270) 

132-  If  the  VI  Corps  went  too  far  inland  it  would  risk  annihilation  (270) 

133.  Allied  intelligence  judged  the  German  strength  as  sufficient  but  not 
overwhelming  (270) 

134.  It  would  seem  that  the  Allied  hesitation  on  the  Anzlo  shore  stemmed 
from  a  belief  in  German  invincibility  (270) 

’35.  This  belief  was  a  product  of  doubt  and  uncertainty  both  before  and 
during  the  operation  (270) 

This  belief  was  used  later  to  explain  the  Inevitability  of  the 
actual  course  of  events  (270) 

137.  The  only  thing  that  disturbed  Lucas  was  the  necessity  to  safeguard 

the  port  (271) 

:  )h.  Without  it  the  swift  destruction  of  the  corps  was  inevitable  (271) 
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139.  Lucai  thousht  he  could  not  havp  Hnnp  rflffprpnt-ly  (771) 

140.  Nevertheless  the  alternative  remained  a  disturbing  possibility 
to  him  (271) 

141.  He  admitted  a  mass  of  armor  and  motorized  infantry  might  have 
reached  the  Alban  Hills  (271) 

142.  He  was  sure  he  could  not  have  remained  there  (271) 

143.  Any  force  that  far  from  Anzio  would  have  been  in  the  greatest 
jeopardy  (271) 

144.  Lucas  did  not  see  how  it  would  have  escaped  annihilation  (271) 

145.  As  it  turned  out  he  believed  he  had  reached  positions  from  which 
the  enemy  was  unable  to  dislodge  him  (271) 

146.  Lucas  believed  the  whole  operation  a  mistake  (271) 

147.  Anyone  who  expected  him  to  push  to  the,  Alban  Hills  was  bound  to 
be  disappointed  (271) 

148.  Lucas  had  never  considered  doing  so  (271) 

149.  He  considered  his  mission  to  be  taking  the  port  and  its  surroundings  (271) 

150.  Perhaps  this  was  an  influence  of  the  Navy  (271) 

151.  Admiral  Cunningham  asserted  no  reliance  could  be  placed  on  over 
the  beaches  maintenance  (271) 

152.  Unfavorable  weather  was  probable  (271) 

153.  General  Clark  said:  "You  can  forget  /this  goddam  Rome  Business"  (271) 

/  * 

1 

154.  The  capture  of  Anzio  was  an  obvious  objective  (271) 

155.  But  early  occupation  of  the  Alban  Hills  was  vitpl  (271) 

156.  The  Anzio  forces  later  realized  the  importance  of  the  hills  (272) 

157.  Was  General  Lucas  justified  in  delaying  seven  days  before  starting 
his  offensive?  (272) 

158.  Could  he  have  gotten  away  with  the  gamble  of  an  immediate  drive  to 
the  Alban  Hills  (272) 

159.  Certainly  the  complete  surprise  achieved  at  the  landing  could  have 
bean  exploited  (272) 
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160.  According  ro  Tenth  Army  estimates  only  a  quick  cutting  of  lines  of 
communication  would  have  led  to  major  Allied  success  (272) 

161.  Such  a  success  would  be  more  likely  to  capture  Rome  (272) 

162.  According  to  Kesselring's  Chief  of  Staff,  an  audacious  flying 
column  could  have  penetrated  to  the  city  (272) 

163.  He  was  astonished  at  the  Allied  passivity  (272) 

164.  Could  the  Germans  have  withstood  a  dynamic  front  as  they  did  the 
static  front?  (272) 

165.  Would  they  have  dared  to  hold  both  at  Anzio  and  at  Garigliano  (272) 

166.  An  Allied  force  ensconced  on  the  Alban  Hills  would  have  been  a 
much  greater  threat  chan  those  or  Anzio  (272) 

167.  The  answer  can  only  be  speculation  (272) 

168.  Alexander  thought  an  aggressive  commander 'would  have  acted 
differently  than  Lucas  (272) 

169.  He  would  and  could  have  pushed  regimental  strength  patrols  to  the 
hills  (272) 

1/0.  The  shock  of  Allied  troops  directly  threatening  Rome  might  have 
by  itself  permitted  Allied  retention  of  both  the  hills  and  a  supply 
corridor  (272) 

171.  A  bluff  might  have  worked  (272) 

1/2.  General  Patton  might  have  been  successful  (272) 


FROM:  COMMAND  DECISIONS  -  Office  of  the  Chief  of  Military  History 
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EMPIRICAL  BAYES  AND  THE  DESIGN  AND 
anat.vctc  OF  EXPERIMENTS 


Richard  G.  Krutchkoff 
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Blacksburg,  Virginia 


Let  me  start  with  an  introduction  to  Empirical  Bayes.  Consider 
the  simple  estimation  situation  in  which  we  observe  a  value  x  of  the 
random  variable  X  which  has  distribution  function  F(x|9),  and  must 
estimate  9  with  small  squared  error.  In  the  parametric  situation, 
the  form  of  the  distribution  function  is  known  except  for  the  value 
of  the  parameter  9.  Both  x  and  6  may  be  vector  valued. 

For  Empirical  Bayes  to  be  applicable  here  we  consider  the  case 
in  which  the  estimation  problem  is  routiue.  That  is,  we  observe  x^ 

from  FCxjJe^)  and  must  estimate  6^;  then  some  time  later,  in  a  similar 

but  Independent  situation,  we  observe  x^  from  F(x2|e2)  and  muBt  estimate 

02.  This  routine  situation  continues  until  at  present  we  have  the 

observation  x  from  F(x  Is  )  and  wa  must  estimate  6  .  These  estimating 
n  o  n  n 

situations  we  call  experiences.  As  an  example  consider  the  situation 
encountered  at  the  Radford  Arsenal.  Every  six  weeks  base  grain  was 
mixed  and  subsequently  cured  with  Nitroglycerine  in  order  to  form 
propellent  for  the  Nike  missile.  It  was  desired  to  estimate  the  para¬ 
meters  for  each  base  grain  separately  since  it  was  believed  that  the 
parameters  would  vary  from  base  grain  to  base  grain  in  some  unpredictable 

meaner.  Sihce  the  6  values  8.,  9.,  .6,,...,  e  ,  vary  in  an  arbitrary 

a  i  j  n 

and  unpredictable  manner,  we  assume  that  9  is  a  random  variable  but  with 
a  completely  unknown  distribution.  It  is  important  to  note  chat  we  do 
not  use  our  Ignorance  of  the  distribution  as  a  Justification  for  choosing 
a  diffuse  or  uniform  distribution. 

If  one  were'  to  take  a  completely  classical  approach  to  the  problem 

he  would  note  that  X  is  a  sufficient  statistic  for  6  .  This  is  easily 
n  n 

seen  by  noting  that 

n 

F(x1,x2,x3,...,xn|91,02,e3,...,9n)  -  H  F(xi|0i)  . 

i-1 

The  classical  solution  to  the  problem  therefore  muet  completely  Ignore 
the  observations  x^,x2,x3,. . . »*n-^  and  use  only  the  observation  in 

estimating  9fi.  Even  intuitively  this  is  an  unfortunate  result. 
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T!.«  |julc  Bayesian  approach  to  the  problem  assumes  a  form  for 
the  distribution  of  ■,  say  G(tt),  and  then  obtains  the  estimator 

/ndF(xJe)dG(e) 

K(  :V  =  /dF(x"  1 0)dG( 0) 

(the  posterior  mean)  as  the  minimizing  estimator.  If  the  choice  of 
G(  )  is  correct  then  this  is  indeed  the  minimizing  estimator.  If  the 
choice  of  G(d)  is  not  correct  then  the  estimator  may  have  a  very  large 
mean  squared  error.  Note  that  the  Bayes  estimator  Ignores  the  past 
experience  x, ,  x  _,..., x  , ,  as  did  the  classical  estimator.  Surely, 

we  should  be  able  to  use  this  experience  in  some  way. 

The  Empirical  Ba^es  approach  to  this  problem  is  now  very  simply 
stated.  We  find  the  Bayes  estimator  E(0|x  ),  which  is  usually  given 

in  terms  of  the  unknown  distribution  function,  and  express  it  in  a 
form  which  can  be  estimated  from  the  data,  x^,  x^,  x^,...,  xf,  without 

knowledge  of,  or  assumptions  about,  the  unknown  prior  distribution. 

The  proper  forms  fcr  ECojx^)  are  given  in  Rutherford  and  Krutchkoff  [11] 

for  four  general  families  of  distributions.  Examples  of  members  of 
these  general  families  are  the  Poisson,  Negative  Binomial,  Logarithmic, 
Gamma,  Normal  (unknown  mean),  Normal  (unknown  variance),  Exponential, 
and  the  Uniform  Distributions.  In  Lemon  and  Krutchkoff  [5]  an  Empirical 
Bayes  estimating  procedure  is  proposed  for  any  discrete  conditional 
distribution.  This  procedure  has  now  been  extended  to  include  any 
conditional  distribution. 

Let  me  now  briefly  mention  some  recent  applications  of  this  approach. 
First,  consider  the  simple  linear  orthogonal  model 

'ii  -  i  +  a(x1-x)  +  £t 

where  the  errors  are  assumed  to  be  normal  and  where  we  must  routinely 
estimate  i  and  .  This  problem  is  considered  in  Clemmer  and  Krutchkoff  [1] 
and  the  example  analyzed  there  is  worth  rementioning  here. 

Every  six  weeks  Ridford  Army  Arsenal  mixed  Base  Grain  for  their 
Nike  missiles.  The  Base  Grain  was  then  cured  with  Nitroglycerine  to  form 
rocket  propellent.  Estimates  of  the  parameters  in  a  linear  model  were 
required  for  each  Base  Grain.  Since  the  chemicals  were  purchased  at 
different  times  and  mixed  at  different  times  under  different  atmospheric 
conditions,  the  parameters  were  expected  to  vary  in  an  unpredictable 
manner.  Using  the  estimator  for  the  normal  distribution  given  in 
Rutherford  and  Krutchkoff  [11],  Clemmer  and  Krutchkoff  [1]  found  the 
desired  estimators  as 
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E(a|a) 


and 


«  f  (a) 


/«,  * 

where  a  and  6  are  the  usual  Least  Squares  or  Maximum  Likelihood 

2 

estimators  for  a  and  6,  o  Is  the  error  variance,  N  is  the  number  of 
observations  taken  in  the  ntl*  experience,  is  the  usual  sum  of 

squares  of  the  independent  variable  (  J  (X.-X)  );  f (a)  is  the 

i-1  1 

marginal  density  of  the  least  squares  estimator;  and,  f'(a)  is  the 
derivative  of  this  density  estimated  at  the  same  point.  In  general, 
the  form  of  the  estimator  is  simply  the  least  squares  estimator  plus 
a  correction  factor.  The  correction  factor  is  the  variance  of  the 
least  squares  estimator  times  the  ratio  of  the  derivative  of  the 
marginal  density  to  the  marginal  density  itself  evaluated  at  the 
present  value  of  the  least  squares  estimator.  It  is  worth  noting 
here  that  if  the  parameter  has  a  diffuse  prior  distribution  then 
the  ratio  of  the  derivative  of  the  density  to  the  density  will  in 
effect  be  aero  and  the  Empirical  Bayes  estimator  will  be  the  Classical 
estimator.  Thus,  When  ths  prior  information  is  of  little  value  this 
correction  term  disappears  rather  than  biasing  the  result  undully. 


The  estimate  of  the  ratio  recommended  in  the  paper  [1],  can 
be  simply  Vritten  as 
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and  where 


h  *  n 


■1/5 


max 


i  l  (<ir“)2’  £  I  (yi-y> 


-.2 


1-1 


1-1 


with  a 


I  dt ,  and  y  -  i  \  yL  . 
1-1  1-1 


Note  that  we  are  using  to  represent  the  least  squares  estimate 
for  a  in  the  experience  and  ^  Is  to  be  Interpreted  as 

ity.  An  estimate  of  ^  In  obtained  by  simply  replacing 

£  / 


un 


the  a's  with  B's. 


f  <e) 


The  improvement  of  this  Empirical  Bayes  estimating  procedure  is 
then  compared  with  the  Classical  procedure  by  taking  a  ratio  of  the 
mean  squared  errors.  This  was  dona  by  choosing  a  distribution  for  a 
and  3  and  generating  31  from  this  distribution  and  then  generating 

several  observed  values  y^  from  the  regression  equation 

yi  "  “l  +  el  +  Ei  wit^  ei  ^e:t,n8  rand°m  normal  errors.  This 

was  done  fifty  times  obtaining  new  values  for  the  observations  at  each 
experience.  The  Empirical  Bayes  estimator  was  obtained  using  the  i-1 
previous  sets  of  data  aa  past  experience.  The  entire  run  of  50 
experiences  was  then  repeated  500  times.  The  average  ratio* of  the 
Empirical  Bayes  squared  errors  to  the  Classical  variance  was  then 
plotted  as  a  function  of  the  number  of  experiences.  This  was  then 
repeated  for  many  different  prior  distributions,  error  variances 
and  experimental  designs.  It  was  found  that  the  mean  squared  error 
for  the  Empirical  Bayes  procedure  was  never  greater  than  that  of  the 
classical  procedure  with  the  ratio  of  the  Empirical  Bayes  mean  squared 
error  to  the  Classical  mean  squared  error  often  dropping  well  below 
unity.  It  was  also  determined  that  the  ratio  of  the  mean  squared 
errors  depends  not  on  the  prior  distribution  or  the  error  variance 
or  the  design  but  solely  on  one  relation  Involving  them;  namely, 


Z 


Var  (a | a) 


Var  a 
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for  n  and  a  similar  expression  for  8.  This  index  is  the  ratio  of 
i-lie  least  uquaies  variance  to  uie  variation  in  the  parameter.  intui¬ 
tively,  if  the  least  squares  variance  is  small  and  the  parameter 
variation  large  not  much  information  can  be  extracted  from  past  ex¬ 
perience.  This  is,  in  fact,  the  case.  When  Z  is  extremely  small, 
below  0.1,  we  find  that  the  derivative  of  the  marginal  on  r  is  small 
compared  with  the  density  itself  and  the  correction  factor  disappears. 

On  the  other  hand,  when  the  least  squares  variance  is  large  and  the 
parameter  variation  small,  much  Is  to  be  gained  from  past  experience. 
However,  when  Z  is  very  large  (say  10)  then  one  might  as  well  assume 
that  the  parameter  is  not  varying  at  all  and  pool  all  the  data.  The 
interesting  and  realistic  range  is  when  Z  is  about  unity.  Figures  1, 

2,  and  3  given  here  are  for  Z  values  of  0.5,  1  and  2  with  past  experience 
ranging  from  one  to  fifty.  The  solid  line  is  for  known,  obtained  by 
pooled  data  or  estimated  from  the  present  data  with  N  j>  20.  The  broken 
line  is  for  o?  estimated  from  the  present  set  of  data  with  N  ■  8.  The 
reduction  in  mean  squares  error  obtained  from  nineteen  batches  of  Base 
Grain  is  given  here  iu  Table  1. 

Then  In  Martz  and  Krutchkoff  [6]  the  regression  model  was  extended 
to  the  multilinear  model 


y^  ■  a  +  8X^  +  +  ...  +  e^ 

where  orthogonality  was  not  required.  This  required  obtaining  the 
multivariate  extension  to  the  estimators  presented  in  CLemmer  and 
Krutchkoff  [1]  and  finding  estimators  for  joint  marginal  densities  and 
their  vector  derivatives.  The  mean  squared  errors  once  again  were 
never  greater  than  thoee  of  the  .least  squares  estimators  with  their 
ratio  often  dropping  well  below  unity  for  the  usual  Z  values. 

2 

The  model  was  than  extended  to  allow  for  the  possibility  that  o 
varies  in  an  unpredictable  way  from  experience  to  experience.  Consider 
the  model 


y  ■  X8  +  c 

where  $  is  a  k<p  matrix,  of  known  fixed  quantities  which  remains  the 

same  from  experiment  to  experiment  and  c  is  distributed  N(0,  ,2I) .  We 

2  '  ” 
assume  §  and  a  vary  randomly  from  experiment  to  experiment  according 

to  the  unknown  prior  distribution  G(g,o^). 

2 

If  X  is  of  rank  p,  the  usual  least  squares  estimators  for  8  and  -j 
are  : 


B  -  (X’xr1  X’y 
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and 


2  (X  -  Xg)  '  (?  -  X§) 

■  .  . .  •  '  .1  ■■■■  HI.  ■  — . . . 

k-P 


*2 

Denote  (k-p)a  by  S.  For  this  situation,  the  Empirical  Bayes  estimators 
are  given  by 


Stf'X)"1  f  .  ,  k-p-2(?*S) 
6  -  8  +  -  -Safi - — - - 

k’P"2  fN,k-p<!'S> 


and 

•2  .  1  S  W-ztM) 
jk->-2 

I 

The  past  experience  for  this  is  in  the  form  of  the  vectors 


The  ratio  of  densities  given  here  are  estimated  in  a  way  similar  to 
the  expressions  already  given  but  a  bit  more  complicated.  The  actual  ' 
formulas  are  not  yet  published,  but  can  be  found  in  the  Virginia 
Polytechnic  Institute  Ph.D.  dissertation  of  one  of  my  students  (see 
Reneher  [8]).  Needless  to  say,  many  simulations  were  run  and  never 
was  the  Classical  squared  error  for  any  component  of  8  smaller  than 
that  of  the  Empirical  Bayes  procedure  for  as  few  as  one  past  experience. 
The  amount  of  improvement  was  similar  to  the  figures  already  showp.  On 
the  other  hand,  there  were  cases  in  which  we  needed  as  many  as  five 
experiences  before  the  Empirical  Bayes  procedure  had  a  smaller  squared 
error  than  the  Maximum  Likelihood  procedure  when  estimating  c 2.  See 
for  example  figure  4. 

Another  example  of  an  Empirical  Bayes  application  is  in  Sequential 
Estimation.  Here  we  considered  the  case  in  which  one  must  sequentially 
estimate  the  mean  of  a  Normal  distribution,  the  cost  being  the  sum  of 
the  mean  squared  error  and  a  constant  times  the  number  of  observations 
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taken.  Although  the  big  problem  was  the  stopping  rule,  we  had  some 
difficulty  in  handling  th«  p<u>i.  experience,  since  the  number  of 
observations  taken  differed  from  experience  to  experience.  This 
problem  was  solved,  however,  and  the  solution  is  generally  applicable 
to  this  type  of  past  experience.  The  results,  i.e.,  the  ratio  of 
Empirical  Bayes  cost  to  Classical  cost  plotted  as  a  function  of  the 
number  of  experiences  is  typically  as  shown  in  the  solid  line  of 
figure  5.  The  dotted  line  is  the  improvement  obtained  by  using  the 
Classical  stopping  rule  and  then  the  Empirical  Bayes  estimator.  Since 
determining  the  stopping  time  by  the  Empirical  Bayea  approach  is  ao 
vary  tedious  we  recommend  using  this  hybrid  approach.  Unfortunately, 
the  details  of  thess  procedures  have  not  as  yet  been  submitted  for 
publication.  They  are  available,  however,  in  the  Ph.D.  dissertation 
of  another  one  of  my  students  (see  Lemon  14]). 

Another  project  presently  underway  is  the  estimation  of  the 
power  spectral  density  function  in  a  time  aeries.  In  a  time  series 
situation  one  often  has  peat  experience  from  similar  situations  or 
one  can  break  the  present  time  series  into  pares  which  can  be  considered 
experiences.  Tor  example,  in  tasting  atraaa  on  airplane  wings  in  a  wind 
tunnel  one  has  the  results  of  tests  on  other  wings.  When  the  Navy  obtains 
a  time  aeries  signal  from  tha  path  of  a  submarine,  it  la  merely  one  exper¬ 
ience  in  many  such  experiences.  In  each  of  these  tha  object  ia  to  obtain 
the  power  amplitude  of  tha  various  frequency  components.  Vs  have  used 
the  Empirical  Bayas  approach  tp  obtain  efficiencies  of  the  order  of  1501 
that  of  tha  standard  approach.  This  work  is  still  in  progross. 

A  project  which  is  just  about  complete  now  involves  estimating  the 
^arrival  and  aarvica  'parameters  in  a  Qua.  Va  have  estimators  for  Quaa 
involving  tha  exponential  and  tha  Ehrlang  distributions.  As  usual,  the 
Empirical  Bayas  estimators  have  a  significantly  smaller  mean  aquared ' 
error  than  tha  usual  estimators,  A  typical  example  is  depicted  in 
Figure  6.  A  is  tha  mean  arrival  time,  p  the  mean  service  time,  and  p 
la  tha  traffic  intensity  for  an  M/M/1  Qua. 

.  i  • 

Now  let  us  discuss  the  Analysis  of  Variance.  First,  vs  considered 
the  random  affects  modal: 


»lj- 

wltli  I  effects  and  J  repetitions  per  effect.  We  were  able  to  estimate 
the  variance  a*  of  the  effects  as  well  as  the  error  variance  by  using 

peat  axparianca.  By  making  a  ratio  of  these  two  statistics,  wa  came 
up  with  an  analog  to  tha  F  statistic.  The  percentage  points  of  this 
statistic  ware  found  by  Monte  Carlo  simulation  for  several  values  of 
I,  J  and  numbers  of  experiences.  We  found  that  these  tables  depend 
only  on  J  and  N  and  not  on  I.  Many  typical  situations  were  then 
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simulated  and  the  power  for  the  Emnlrlral  R»y««  test  of  the  Hypothesis 
3A  -  0  was  always  significantly  greater  than  that  of  the  usual  F-test 

for  the  same  size,,  After  as  few  as  ten  or  fifteen  past  experiences 
the  power  was  as  much  as  50-802  higher  for  the  Empirical  Bayes  teat. 

For  the  fixed  effects  model 


y  ■  xa  +  c 

we  had  to  reparameterize  to  full  rank  before  proceeding.  Once  this 
was  done,  the  estimates  were  the  same  as  for  the  linear  regrension 
situation.  In  order  to  test  the  Hypothesis  g  ■  0,  we  made  an  analogy 
to  the  F  statistic  by  using  the  sum  of  squares  of  the  Empirical  Bayes 
estimators  for  the  parameters  in  the  numerator  and  the  estimated  error 
variance  in  the  denominator.  Here  the  percentage  points  were  found  to 
depend  on  the  number  of  repetitions,  the  number  of  experiences  and  also 
the  number  of  effects.  Only  tables  for  up  to  six  effects  were  simulated. 
Once  ugaln  the  Empirical  Bayes  test  always  more  ppwerful  than  the 
usual  F-test  (when  there  were  at  least1  four  past  experiences).  Un¬ 
fortunately,  the  details  of  this  topic  are  not  yet  in  print,  but  can 
be  found  in  Rencher  [ 8] I  j 

t 

Before  leaving  parametric  Empirical  Bayes,  I've  been  asked  to 
briefly  mention  the  results  we  obtained  in  long  range  prediction  of 
rainfall.  The  Weather  Bureau  puts  out  a  map,  twice  a  month,  predicting 
rainfall  in  the  categories  of  light,  moderate  and  heavy  for  a  period 
of  30  days,  the  predictions  are  for  large  areas  and  not  for  particular 
locations.  We  were  asked  to  use  this  map  and  predict  for  each  city  the 
amount  of  rainfall  within  the  next  30  days.  We  were  able  to  do  juat 
this.  We  found  a  procedure  for,  predicting  the  probability  distribution 
of  rainfall  in  Inches  for  any  location  that  had  been  collecting  such 
data  for  at  least  fifteen  years.  The  results  were  remarkably  successful. 
One  could  only  compare  with  the  Weather  Bureau,  however,  for  the  categories 
light,  moderate,  and  heavy,  The  Weather  Bureau,  for  example,  was  correct 
in  Roanoke,  Virginia,  but  302  of  the  time  while  we  ware  correct  more  like 
702  of  the  time.  The  details  of  this  project  can  be  found  in  Philpot 
and  Krutchkof f  [ 7) .  ' 

Let  us  now  turn  to  another  type  of  Empirical  Bayes  Estimation. 

Consider  the  situation  where  the  distribution  of  the  observation  is  it¬ 
self  unknown;  a  non-parametric  situation.  Hfcre,  we  observe  the  value  x 
ot  the  random  variable  X  whose  distribution  depends  in  some  unknown  way 
on  "  and  we  are  asked  to  estimate  &  with  small  squared  error.  Such 
estimation  is  of  course  impossible.  For  this  situation,  we  assume  that 
there  is  a  supplementary  observation  obtained  after  the  estimate  is 
j?iven,  perhaps  in  the  form  of  customer  feedback.  To  be  more  specific, 
let.  us  say  we  have  observed  an  x  from  some  unknown  distribution  and 
must  estimate  the  “  value  related  to  it  in  sope  unknown  way.  Later, 
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we  are  given  an  observation  y  from  the  random  variable  Y  whose 
distribution  tiny  likewise  be  unknown  but  for  which  RV  *  fl-:  Thar 
is,  our  supplementary  sample  is  an  unbiased  estimate  of  6.  Un¬ 
fortunately,  this  estimate. is  too  late.  In  Krutchkoff  [3]  the 
problem  is  assumed  to  be  routine  with  the  a  values  varying  in  an 
unpredictable  way.  The  Empirical  Bayes  estimating  procedure  for 
this  situation  is  very  simple.  If  the  present  observed  values  is 
x  and  there  are  several  past  experiences  with  this  same  value  of 
x,  then  use  as  your  estimate  the  average  of  the  supplementary  values 
y  which  occurred  after  the  occurrence  of  the  value  x.  If  there  la 
not  a  sufficient  number  of  past  experience  at  x.  -  x,  then  make  a 

1  •'  \ 

linear  regression  using  the  past  y  values  as  the  dependent  variable 
and  the  past  x  values  as  the  independent  variable  and  find  the  re¬ 
gression  value  of  y  at  x.  The  results  of  such  a  situation  are  given 
in  Krutchkoff  [3].  Generally,  after  a  few  past  experiences  the 
Empirical  Bayes  mean  squared  error  drops  below  the  mean  squared  error 
of  the  Classical  estimator  which  would  be  used  if  the  distribution 
of  X  were  actually  known.  Here  we  have  not  only  au  estimator  which 
we  can  use  when  nothing  else  exists,  but  one  which  ie  better  than  the 
usual  estimator  when  one .does  exist. 

An  extension  of  this  non-par ame trie  approach  was  given  in 
Gabbert  and  Krutchkpff  [2].  Here  we  assumed  that  a  machine  producing 
items  w*s  to  be  checked  to  determine  when  it  wae  Out  of  Control.  The 
linear  regression  form  of  the  estimate  was  employed  but  using  only  the 
past  fifteen  axperiencaa. 

A  sample  of  defectives  waa  taken  and  x,  the  sample  proportion  of 
defectives  found.  The  value  y  was  later  supplied  by  some  other  pro¬ 
cedure  such  that  EY  -  p,  the  true  proportion  of  defectives.  Clearly, 
each  box  la  an  experience  with'' the  true  proportion  of  defectives 
varying  randomly.’  The  estimate  of  the  present  proportion  wae  obtained 
from  x  by  using  the  past  fifteen  values  of  x,y,  in  a  linear  regression;', ’ 
and  obtaining  the  regression  value  of  y  for  the  prehent  value  of  x.‘ 

The  variance  needed  in  the  control  chert  wee  also  obtained  ftp m  the 
linear  regression  as  the  variance  of  the  regression  line  at  the 
present  value  of]x.  A  typical  power  function  for  the  Empirical  Beyee 
procedure  ie  giv4n  in.  figure  7.  Here  the  machine  was  caused  to  go  out 
of  control,  producing  a  proportion  of  defectives  varying  randomly  about 
some  undesirable  proportion  (PQ  is  .ehs  In  control  value).  The  power 

for  the  procedure  la  seen  to  start^out  below  the  usual  control  chert 
procedure  but  after  a  few  experiences  with  the  out  of  control  machine 
the  power  increases  rapidly.  In  this  example,  the  Empirical  Bayes 
procedure  detects  the  out  of  control  situation  at  about  the  twelfth 
consecutive  sample  whereas  it  takes  the  usual  procedure  about  18 
samples.  Of  course,  since  this  Empirical  Bayas  procedure  does  not 
make  use  of  the  fact  that  the  sample  is  Binomial,  it  can  be  applied 
in  situation!  where  the  distribution  of  the  first  sample  is  itself 
unknown.  We  ere  presently  working  on  a  single  sample  non-parametric 
approach  but  not  enough  results  are  as  yet  available  to  present  anything 
here. 
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Although  the  title  of  this  paper  contains  the  "design  of 
experiments I  have  virtually  nothing  as  vet  to  report.  Several 
results,  however,  can  easily  be  predicted.  For  the  non-parametric 
supplementary  sample  situation  we  did  not  require  the  present  sup¬ 
plementary  sample  to  determine  the  estimator  or  its  squared  error. 

It’  the  squared  error  is  within  tolerable  limits,  we  need  not  take  the 
supplementary  sample  at  all.  We  can,  in  effect,  calibrate  the  pre¬ 
liminary  sample  eliminating. the  need  for  talking  the  more  costly 
supplementary  samples. 

In  the  parametric  situation  we  recall  that  our  squared  error 
was  smaller  thin  that  of  the  Classical  procedure.  By  estimating 
the  prior  variance  we  can  estimate  the  efficiency  of  this  procedure 
and  thus  be  able  to  predict  the  number  of  observations  one  needs  to 
obtain  a  predetermined  squared  error.  This  number  will,  of  course, 
be  smaller  than  that  required  by  the  Classical  Procedure. 

No  doubt  there  are  optimal  designs  for  the  Empirical  Bayes 
procedures.  Since  Empirical  Bayes  is  more  efficient  than  the 
Classical  approach  using  the  classically  optimal  design  it  makes 
good  sense  to  hope  for  an  even  better  efficiency  when  we  find  the 
Empirical  Bayes  optimal  design.  This  question  la,  as  yet,  unanswered, 
but  we  are  working  on  it. 


REFERENCES 

1.  Clemmer ,  B.  A.  and  Krutchkoff,  R.  G.  (1968).  The  use  of  Empirical 
Bayes  estimators  in  a  linear  regression  model,  BIOMETRIKA,  Vol.f 55, 
pp.  525-534. 

2.  Gabber t ,  J.  T.  and  Krutchkoff,  R.  G.  (1970).  Supplementary  Sample 
Non-parametric  Empirical  Bayes  in  a  Quality  Control  Situation, 
BIOMETRIKA,  to  appear. 

3.  Krutchkoff,  R.  G.  (1967).  A  supplementary  Sample  non-parametric 
Empirical  Bayes  approach  to  some  statistical  decision  problems , 
BIOMETRIKA,  Vol.  54,  pp.  451-458. 

4.  Lemon,  Glen  H.  "Empriical  Bayes  Analysis  of  Some  Sequential 
Experiments."  Virginia  Polytechnic  Institute  Ph.D.  Dissertation, 
December,  1968. 

).  Lemon,  G.  H.  and  Krutchkoff,  R.  G.  (1969).  "An  Empirical  Bayes 
Smoothing  Technique."  BIOMETRIKA,  Vol.  56,  pp.  361-365. 

h .  Martz,  H.  F.  and  Krutchkoff,  R.  G.  (1969).  Empirical  Bayes 
estimators  in  a  multiple  Linear  regression  model.  BIOMETRIKA, 

Vol.  56,  pp.  367-374. 


588 


7.  Philpot,  J.  W.  and  Krutchkoff,  R.  G.  (1969).  Probability  Forecasts 
of  30-Day  Precipitation.  Published  by  the  Water  Resource?  Research 
Center  of  Virginia,  Bulletin  #2,  145  pages. 

8.  Rcncher,  Alvin  C.  "The  Empirical  Bayes  Approach  to  the  Analysis 

of  Variance  and  Linear  Regression."  Virginia  Polytechnic  Institute 
Ph.D.  dissertation,  August,  1968. 

9.  Rutherford,  J.  A.  and  Krutchkoff,  R.  G.  (1967).  The  Empirical  Bayes 
approach:  Estimating  the  prior  distribution,  BIOMETRIKA,  Vol.  54, 
pp.  326-328. 

10.  Rutherford,  J.  R.  and  Krutchkoff,  R.  G.  (1969).  (asymptotic 
optimality  of  Empirical  Bayes  estimators,  BIOMETRIKA,  Vol.  56, 

pp.  220-222.  | 

'  *  I  c 

11.  Rutherford,  J.  R.  and  Krutchkoff,  R.  G.  (1969).  Some  Empirical 

Bayes  techniques  in  point  Estimation,  BIOMETRIKA,  Vol.  56,  pp.  133-137. 


589 


TABLE  1 


NIKE  MISSILE  RESULTS 


MAXIMUM  PRESSURE  VS.  AGE: 

FOR  as  FOR  0: 


n 

Z 

R 

Z 

R 

2 

.43 

.98 

1.34 

.93 

3 

.51 

.95 

1.72 

.88 

4 

.45 

.95 

1.08 

.87 

5 

.23 

.97 

.53 

.90 

6 

.45  . 

.91 

.89 

.85 

7 

.42 

.91 

.98 

.83 

8 

.74 

.86 

1.38 

.75 

9 

.64 

.88 

1.30 

.75 

10 

.59 

.86 

1.23 

.72 

11 

1.04 

.77 

5.01 

.58 

12 

.95 

.77  1 

3.69 

.56 

13 

.30 

.91 

.71 

.81 

14 

.69 

.81  ! 

1.72 

.80 

15 

.54 

.86 

1.36 

.70 

16 

.54 

.86 

1.57 

.69 

17 

.41 

.89 

,2.51 

.61 

18 

.52 

.86 

1.57 

.69 

19 

.17 

.96 

.45 

.87 
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Figure  i 

Ratio  of  mean  squared  error  of  the  Empirical  Bayes  estimator  to 
tte  MSE  of  the  least  squares  estimator  for  Z  = .  5 


Ratio  of  the  mean  squared  error  of  the  Empirical  Bayes  estimator  to  the  mean  squared 
error  of  the  least  squared  estimator  for  Z  =  2 


Figure  4 

ared  error  of  the  Empirical  Bayes  Estimators  to  the  MSE 
maximum  likelihood  estimators  for  „  0t  ,  and  2 
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Ratio  of  cost  of  Empirical  Bayes  procedure  to  the  cost  of  the  Classical 
Procedure  in  sequential  estimation 
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Presentation  made  by  ,! 

\  '■  ‘  \ 


Dr.  Frank  E.  Grubbs 
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YOUDEN  AWARDED  THE  1969  SAMUEL  S.  WILKS  MEMORIAL  MEDAL 


Dr.  W.  J.  Youden,  now  retired  from  the  National  Bureau  of  Standards, 
lias  been  awarded  the  Samuel  S.  Wiika  Memorial  Medal  for  1969.  The  an¬ 
nouncement  of  Dr.  Youden's  selection  for  the  1969  Wilks  Award  was  one 
of  the  highlights  of  the  Fifteenth  Annual  Conference  on  the  Design  of 
Experiments  in  Army  Research,  Development  and  Testing,  which  was  held 
at  the  U.S.  Army  Missile  Command,  Huntsville,  Alabama,  22-24  October 
1969.  Dr.  Youden  has  long  been  recognised  as  one  of  the  outstanding 
applied  statisticians  by  both  the  U.  S.  A.  and  countries  abroad,  as  well, 
having  made  many  fundamental  contributions  to  the  design  and  analysis  of 
statistical  experiments  and  methodology.  The  citation  for  Dr.  Youden 
reads  as  follows: 

To  Dr.  W.  J.  Youden,  father  of  'Youden  Squares'  and  the 
'Youden  Diagram,'  for  his  extensive  contributions  to  the  art 
and  practice  of  experimentation  in  the  sciences  and  engineering, 
through  conception  and  lucid  exposition  of  novel,  yet  rather 
elementary,  techniques  of  statistical  analysis  and  crafty  ap¬ 
plication  of  standard  methods;  and  through  his  exceptional 
productivity  as  an  author,  indefatigable  energy  and  phenomenal 
effectiveness  as  a  speaker,  by  which  he  has  inspired  a  whole 
generation  of  scientists  and  anglneera  to  greater  achievements 
through  application  of  his  unique  statistical  precepts. 

Previous  recipients  of  the  Samuel  S.  Wilks  Memorial  Medal  include: 

John  W.  Tukey,  of  Princeton  University  (1965);  Major  General  Leslie  E. 

Simon  (1966);  William  G.  Cochran  of  Harvard  University  (1967);  and, 

Jerzy  Neyman  of  the  University  of  California  (1968) . 

The  Samuel  S.  Wilks  Memorial  Medal  Award  is  administered  by  the 
American  Statistical  Association,  a  non-profit,  educational  and  scientific 
society  founded  In  1839.  The  Wilks  Award  is  given  each  year  to  a  statis¬ 
tician  and  is  based  primarily  on  his  contributions  to  the  advancement  of 
scientific  or  technical  knowledge  in  Army  statistics,  ingenious  application 
of  such  knowledge,  or  successful  activity  in  the  fostering  of  cooperative 
scientific  matters  which  coincidentally  benefit  the  Army,  the  Department 
of  Defense,  the  U.S.  Government,  end  our  country  generally. 

The  Award  consists  of  a  medal,  with  a  profile  of  Professor  Wilks 
and  the  name  of  the  Award  on  one  side,  the  seal  of  the  American  Statistical 
Association  and  name  of  the  recipient  on  the  reverse,  and  a  citation  and 
honorarium  related  to  the  magnitude  of  the  Award  funds.  The  annual  Army 
Design  of  Experiments  Conferences,  at  which  the  Award  is  given  each  year, 
are  sponsored  by  the  Army  Mathematics  Steering  Committee  on  behalf  of  the 
Office  of  the  Chief  of  Research  and  Development,  Department  of  the  Army. 

The  funds  for  the  S.  S.  Wilks  Memorial  Award  were  donated  by 
Philip  G.  Rust,  Thomasville,  Georgia. 
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t  f-.t*  approval  n  f  President  A.  Ross  Eckler  ot  the  American 
:  iti-.t.  ii  V  :>  u  litluij,  the  Wilks  Memorial  Medal  Committee  I  or  19h9 

i'roi'ej.-iur  Robert  E.  Bechhofer  -  Cornell  University 

f:  r  a  lessor  wiiiiam  b.  cocr.ran  -  Harvard  University 

Ur.  Francis  C.  Dressel  -  Duke  University  and  the  Army  Research 

Off ice-Durham 

Dr.  Churchill  Elsenhart  -  National  Bureau  of  Standards 
Professor  Oscar  Kempthorne  -  Iowa  State  University 
Dr.  Alexander  M.  Mood  -  University  of  California 
Major  General  Leslie  E.  Simon  -  Retired 
Dr.  John  W.  Tukey  -  Princeton  University 

Dr.  Frank  E.  Grubbs,  Chairman  -  U.  S.  Army  Aberdeen  Research  and 

Development  Center,  Aberdeen  Proving 
Ground,  Maryland 

1. 1 OGKAPHICAL  SKETCH.  Dr.  Youden  was  born  in  Townsville,  Australia, 
on  j’Spi  il.  12,  1900.  Two  years  later  his  father  returned  to  his  birthplace, 

Dover,  England,  with  his  wife  and  young  son;  and  the  three  resided  there 

April  1.902  -  June  1907.  During  these  years  a  sister,  Dora  Alice,  and 
brother,  Harry,  were  born.  In  1907,  the  family  of  five  set  out  for 
America,  and  entered  the  United  States  through  the  Port  of  New  York  in 
July  1907.  They  lived  for  a  while  at  Ivoryton,  Connecticut,  and  at 
Niagara  Falls,  New  York,  where  Jack  attended  the  local  public  schools; 
then  they  moved  to  Rochester,  New  York,  in  1916,  for  Jack's  senior  year 

'ot  high  school.  Youden  spent  the  years  1917-1921  at  the  University  of 

Rochester,  except  for  one  brief  interruption  to  serve  his  new  country 
as  a  private  in  the  U.  S.  Army,  October  15  -  December  12,  1918.  At  the 
University  of  Rochester,  Jack  was  elected  to  the  National  Phi  Beta  Kappa 
honor  society,  and  was  awarded  a  B.  S.  in  Chemical  Engineering  in  June 
1921  The  following  academic  year,  1921-22,  he  continued  at  the  University 
ot  Rochester  as  an  instructor  in  Chemistry,  then  went  the  two  succeeding 
years,  1922-24,  to  Columbia  University  as  a  graduate  fellow  in  chemistry, 
earning  an  M.  A.  (Chemistry)  in  1923;  and  a  Ph.D.  (Chemistry),  in  1924. 

Immediately  following  receipt  of  his  doctorate,  Dr.  Youden  joined 
the  staff  of  the  Boyce  Thompson  Institute  for  Plant  Research  in  Yonkers, 

Mew  York,  as  a  Physical  Chemist.  He  continued  with  the  Institute  in  this 
capacity,  with  two  short  leaves  of  absence  and  one  3-year  assignment  as 
m  Operations  Analyst  with  the  Army  Air  Force,  until  he  Joined  the  staff 
1  the  National  Bureau  of  Standards  in  May  1948  as  Assistant  Chief  of  the 
t at ;  ;  t i cal  Engineering  Laboratory,  which  was  then  beginning  its  second 
■car  of  existence. 


Dr.  Youden  was  often  heard  telling  a  "client"  in  consultation  on 
statistical  aspects  of  experimentation,  or  an  audience  at  one  of  his  well 
j’l-nied  lectures  on  statistical  methodology,  that  he  is  a  "chemist," 

;  mpiyi ,  it  would  appear,  that  he  is  really  not  a  statistician.  Well, 
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Youden  may  have  been  .1!  !  .Iwmist  t-ji  ,  1  h  fir'll  srvn  yens  at  the 
Boyce  Thompson  Institute,  hut  by  September  19’!  he  had  already  begun  to 
dish  out  advice  on  the  statistical  aspects  o!  •■xner  imenf-aM  on  The 
evidence  Is  :o  be  found  In  hts  paper  entitled,  "A  Nomogram  for  use  in 
connection  with  Gutzelt  arsenic  determinations  on  apples,"  published 
In  Vr»l ,  3,  No.  3  of  the  Contributions  ftuu  Lite  Buyce  Thompson  institute, 
pp.  363-374.  And  from  impeccable  authority  we  learn  that  during  the 
academic  year  1931-32  he  commuted  on  his  own  volition  from  Yonkers  to 
Morningside  Heights  in  New  York  City  to  attend  Professor  Harold  Hotelling's 
lectures  on  "Statistical  Inference"  at  Columbia  University.  H*  was  on  hia 
way  to  becoming  an  expert  on  statistical  aspects  of  experimentation.  From 
then  on  he  became  more  and  more  of  a  statistician. 

The  paper  that  was  ultimately  to  make  his  name  a  laboratory,  if  not 
a  household  word,  saw  publication  in  early  1937:  "Use  of  Incomplete 
Block  Replications  in  Estimating  Tobacco  Mosaic  Cirus"  (Contributions 
from  Boyce  Thompson  Institute,  Vol.  9,  No.  1,  pp.  41-48).  Here  he 
gave  examples  and  illustrated  the  application  of  a  new  class  of  symmetrical 
balanced  incomplete  block  designs  that  possessed  the  characteristic  "double 
control"  of  Latin  square  designs,  without  the  restriction  that  the  number 
of  replications  of  each  "treatment"  (or  "variety")  must  equal  the  number 
of  "treatments11  (or  "varieties").  This  paper  and  its  new  designs  led  to 
Dr.  Youden  obtaining  a  ftockfeller  Fellowship  that  enabled  him  to  take  his 
first  leave  of  absence  from  Boyce  Thompson,  and  to  devote  the  academic 
year  1937-38  to  further  work  in  the  field  of  experiment  design  under  the 
direction  of  R.  A.  Fisher  himself  at  the  Calton  Laboratory,  University 
College,  London.  Youden* s  new  rectangular  experiment  designs,  termed 
"Youden  Squares"  by  Fisher  and  Yates  in  the  introduction  to  the  first 
edition  of  their  Statistical  Tables  for  Biological  Agricultural  and  Medical 
Research  (1938) ,  were  found  immediately  to  be  of  broad  utility  in  biological 
and  medical  research  generally;  applicable  but  of  less  value  in  agricultural 
field  trials;  and  with  the  coming  of  World  War  II,  Youden  Squares  proved  to 
be  of  great  value  in  the  scientific  and  engineering  experimentation  connected 
with  the  research  development  activities  of  the  war  effort  of  the  British 
and  their  allies. 

Following  Pearl  Harbor,  Dr.  Youden  took  a  somewhat  longer  leave  of 
absence  from  the  Boyce  Thompson  Institute  to  serve  as  an  Operations  Analyst 
with  the  United  States  Army  Air  Forces,  1942-45,  first  aa  head  of  the 
Bombing  Accuracy  Section  of  the  Operations  Analysis  Unit  of  the  U.  S. 

Eighth  Air  Force  in  Britain,  where  he  directed  a  group  of  civilian  scientists 
seeking  to  determine  the  controlling  factors  in  bombing  accuracy;  then.  In 
the  latter  part  of  World  War  II,  he  was  transferred  to  the  Pacific  to 
conduct  similar  studies  preparatory  to  the  B-29  assult  on  Japan.  Stories 
are  legion  among  the  members  of  the  Operations  Research  Group  of  the 
U.S.A.A.F.  Eighth  Bomber  Command  about  Dr.  Youden's  exceptional  skill  in 
the  invention  of  novel  and  the  adaptation  of  standard  statistical  tools  of 
experiment  design  and  analysis  to  cope  with  various  problems  arising  in 
these  studies  of  bombing  accuracy.  Some  of  these  military  applications 
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-ritten  up  tor  1  mme.l  i  at  *■  use,  ami  embalmed  tor  posterity  in  his 

:  ..ii',  H.’w  to  imptov,-  Formation  Bombing,"  Air  Forte  Manual  No,  67, 
i.io  iii  tiie  imnt  material  to  his  "Bombing  Chart,”  /ir  Forte 
"■...’ill  No.  April  1^45.  He  was  awarded  the  Medal  of  Freedom  in  1946 

his  important  cont r ibutions  to  the  allied  victory. 

in  1947  Dr.  Youden  took  his  third  and  final  leave  of  absence  from 
the  Boyce  Thompson  Institute:  from  May  to  November  1947  he  was  employed 
by  Project  RAND,  Douglas  Aircraft  Company,  Santa  Monica,  California,  aa 
.i  nonsuit  on  statistical  problems  in  design  and  use  of  military  aircraft. 

As  stated  earlier,  Dr.  Youden  joined  the  staff  of  the  National 
Bureau  of  Standards  on  May  10,  1948,  as  Assistant  Chief  of  the  Statistical 
F.ngineering  Laboratory,  Applied  Mathematics  Division.  Three  years  later 
he  became  a  Consultant  (on  statistical  design  and  analysis  of  experiments) 
to  the  Chief,  Applied  Mathematics  Division,  a  post  that  he  held  until  his 
tireraent  on  June  30,  1965.  Since  then  he  has  enjoyed  the  privileges  of 
o  (iuest  Worker  at  the  N'BS. 

During  Dr.  Youden's  first  two  years  at  the  NBS,  a  fraction  of  his 
salary  was  underwritten  by  the  Research  and  Development  Division,  Office 
of  the  Assistant  Chief  of  Staff,  G-4,  Department  of  the  Army.  This  in¬ 
volved  coordination  with  Dr.  Merrill  M.  Flood  and  others  at  the  headquarters 
office  in  the  Pentagon;  visits  to  Dr.  Ellis  Johnson's  group  at  Ft.  McNair; 
and,  quite  characteristically,  Dr.  Youden  took  a  number  of  trips  to  Army 
research  and  development  installations  in  various  parts  of  the  country,  to 
size-up  "the  problems"  in  their  actual  habitats. 

Dr.  Youden's  first  decade  at  the  National  Bureau  of  Standards  saw  the 
invention  and  publication  of  his  two-sample  chart  for  "Graphical  diagnosis 
of  inter-laboratory  test  results"  (Industrial  Quality  Control.  Vol,  15, 

No.  11,  May  1959),  now  called  the  "Youden  Diagram,"  which  has  proved  to 
be  an  indispensable  tool  in  the  inter-laboratory  test  programs  on  the 
N  itionai  Conference  of  Standard  Laboratories  that  provide  continuing 
•  ■ii-vuil  lance  on  the  central  calibration  programs  of  the  U.  S.  National 
Measurement  System. 

The  early  1960's  saw  Dr.  Youden's  exploitation  of  a  class  of  selected 
incomplete  block  designs  of  block  size  two  for  the  specific  purpose  of 
identification  and  estimation  of  the  effects  of  sources  of  systematic 
nor,  the  central  theme  of  his  paper,  "Systematic  Errors  in  Physical 

'.ant.-,,1'  (Physics  Tidiy,  September  1961). 

N.t  the  least  among  Dr.  Youden's  assets  are  the  effectiveness  with 
.  ..  h  he  communicates  both  in  writing  and  speaking;  his  exceptional 

. r  .duet ivi ty  in  both  areas;  and  the  inspiration  with  which  he  amuses 
i-  readers  and  audiences.  During  Dr.  Youden's  almost  two  decades  at 

National  Bureau  of  Standards  he  was  the  sole  author  of  thirty,  and 
-  mth<  r  of  fifteen  published  research  papers;  the  sole  author  of 
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two  books  and  of  seven  chapters  In  other  books;  and  for  six  years  (1954- 
1959)  he  authored  a  highly  original  bi-monthly  column  on  Statistical 
uesign  in  tne  protessional  Journal,  industrial  Engineering  chemistry. 

(These  columns  have  since  been  brought  together  and  Issued  in  booklet 
form  by  the  American  Chemical  Society  under  the  title,  "Statistical 
Design.")  During  this  same  period,  Dr.  Youden  gave  211  talks  around 
the  country  on  topics  In  Statistical  Methodology  and  Experiment  Design, 
under  125  titles,  the  repetition  of  some  talks  being  by  demand.  In 
addition,  he  made  two  lecture  tours  on  behalf  of  the  American  Chemical 
Society,  addressed  the  NBS  Scientific  Staff  Meeting  twice,  and  was  called 
upon  repeatedly  by  the  Bureau  to  address  special  groups  (e.g.,  high 
school  science  teachers).  Almost  without  exception,  he  was  the  speaker 
most  highly  spoken  of  afterwards  by  such  audiences. 

Dr.  Youden's  first  book,  STATISTICAL  METHODS  FOR  CHEMISTS  (1951)  has 
had  a  sale  of  well  over  fiteen  thousand  copies.  Together  with  his  "column," 
this  book  constitutes  one  of  the  best  sources  of  real-life  examples  of 
effective  applications  of  statistical  principles  and  techniques  in  physical- 

sciences  research  and  development  work. 

As  part  of  the  program  of  the  National  Science  Teachers  Association 
to  place  some  of  the  most  recent  advances  in  science  before  junior  and 
senior  high  school  students.  Dr.  Youden  prepared  one  of  the  NSTA'a  Viatas 
of  Science  Books,  EXPERIMENTATION  AND  MEASUREMENT  (1962).  Aa  of  July  1969, 
this  booklet  has  sold  over  52,000  copies;  and  is  continuing  to  sell  at  the 
rate  of  over  1,000  copies  per  year. 

Dr.  Youden's  total  contribution  to  the  art  and  science  of  statistics 
in  experimentation  is  truly  Impressive.  A  few  weeks  before  Dr.  Youden's 
retirement  from  the  National  Bureau  of  Standards  on  June  30,  1965,  the 
Royal  Statistical  Society  elected  Dr.  Youden  to  Honorary  Fellowship  at 
its  annual  meeting  in  London  on  June  2,  1965.  There  can  be  no  question 
that  Dr.  Youden  la  a  very  deraervlng  recipient  of  the  Samuel  S.  Wilks 
Memorial  Medal  for  1969. 
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Myrna  L.  Toivanen,  Human  Factors  Engineering  and 
Simulation,  Systems  and  Research  Division, 
Honeywell  Inc. ,  Minneapolis,  Minn. 

Bernard  S.  Gurman,  Avionics  Laboratory,  U.S.  Army 
Electronics  Command,  Fort  Monmouth,  N.  J. 

Dr.  Erwin  Riser,  Avionics  Laboratory,  U.S.  Army 
Electronics  Command,  Fort  Monmouth,  N.  .1. 


Summary 


This  paper  describes  the  role  of  systematic  real-time  man -in -the -loop  simu¬ 
lations  in  evaluating  man/machine  performance  of  complex  vehicle  control 
systems.  The  simulated  system  consists  of:  1)  a  hybrid  (digital  and  analog) 
computer  system  which  is  used  to  simulate  the  vehicle  dynamics  and  the 
environmental  conditions  defining  the  system  state,  to  drive  pilot  information 
displays,  and  to  translate  the  pilot's  control  inputs  into  vehicle  responses; 

2)  electro-mechanical  flight  displays  and/or  a  computer -addressed  cathode  - 
ray-tube  (CRT)  display  which  are  used  to  provide  the  pilot  with  the  information 
required  to  perform  the  defined  control  task;  and,  3)  a  fixed-base  control 
station  which  is  configured  to  represent  control  characteristics  of  the  vehicle 
being  studied.  The  important  considerations  in  the  formulation  of  the  experi¬ 
mental  design  and  schedule  are  discussed.  The  performance  measures  used 
to  evaluate  overall  system  performance  are  described.  The  limitations  on  the 
application  of  the  simulation  study  results  to  the  real-world  situation  arc  dis¬ 
cussed.  A  summary  of  the  simulation  mechanization,  methodology,  and  results 
of  a  previous  study  of  helicopter  1FR  formation  flight  system  requirements  is 
provided  as  an  example  of  the  use  of  man-in-the -loop  simulations  to  evaluate 
system  performance. 

This  article  has  been  reproduced  photographical ly  from  the  authors  manuscript. 
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THE  USE  OF  A  HYBRID  COMPUTER  SYSTEM  TO  EVALUATE 
PERFORMANCE  OF  COMPLEX  VEHICLE  CONTROL  SYSTEMS 


Introduction 


The  evaluation  of  the  performance  of  complex  systems  would  be  prohibitively 
expensive  if  it  were  necessary  to  wait  until  the  system  were  built  and  only 
then  test  it  under  actual  environmental  conditions.  The  cost  involved  in  re¬ 
designing  and  rebuilding  the  system  at  this  point  in  its  development  would 
probably  increase  the  cost  of  production  systems  by  at  least  an  order  of  magni¬ 
tude.  If  the  system  is  man-oriented  and  there  is  a  chance  that  system  inade¬ 
quacies  may  result  in  catastrophic  human  injury  or  loss  of  life,  the  risk  of 
system  failure  is  too  great  and  evaluating  system  performance  in  this  manner 
becomes  unthinkable. 

Tests  of  aircraft  or  spacecraft  systems  involve  unusually  high  risks,  both  in 
terms  of  human  life  and  equipment  loss.  It  has  been  necessary,  therefore, 
for  spacecraft  and  aircraft  systems  and  design  engineers  to  develop  alterna¬ 
tive  techniques  for  evaluating  performance  of  such  systems  prior  to  actual 
flight  tests.  Tte  most  comprehensive  and  flexible  technique  which  has  been 
developed  has  been  that  of  computer  simulation. 

A  computer  simulation  to  be  used  for  system  performance  analysis  consists 
of  an  approximate  model  of  both  the  system  and  its  relevant  relationships  with 
its  environment  mechanized  in  the  form  of  computer  programs  (see  Figure  1). 
The  system  and  those  environmental  conditions  which  affect  system  perform¬ 
ance  are  defined  in  terms  of  mathematical  equations.  Then  computer  programs 
are  written  to  mechanize  models  via  appropriate  computers  and  algorithms. 
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The*  simulation  of  a  man-oriented  system  must  include  an  additional  math 
model  for  human  behavior  or  must  a; low  the  human  operator  to  be  included  as 
part  of  the  simulated  system.  In  most  complex  man-oriented  systems  the 
operator's  function  is  a  signi1'  "in*  factor  in  total  system  performance  and  is 
ton  complex  to  lie  represented  •  y  existing  human  behavior  models.  When 
this  is  the  case,  the  human  op.  i  atm  is  included  as  one  of  the  subsystems  of 
the  simulation.  Then  the  simulation  must  include  sensory  cues  to  provide  the 
operator  with  necessary  information  and  physical  apparatus  which  allows  him 
to  perform  his  function  in  the  simuiand  system. 

In  addition  to  being  much  less  expensive  than  actual  system  tests,  analysis  by 
simulation  is  much  more  flexible.  After  a  simulation  is  developed,  it  is 
possible  to  systematically  investigate  system  performance  resulting  from 
changes  in  one  or  a  number  of  the  subsystem  models  or  environmental  condi¬ 
tions  by  merely  changing  the  values  of  the  selected  parameters  in  the  com¬ 
puter  programs.  The  system  design  can  be  changed  and  tested  in  much  less 
time  and  at  relatively  low  cost  compared  to  that  involved  in  rebuilding  and 
reinstalling  hardware  in  the  actual  system.  The  sys'om  can  be  tested  step  by 
step,  analyzing  one  subsystem  while  holdmi/  other  system  parameters  constant, 
adding  to  the  complexity  of  the  model  only  as  greater  levels  of  detail  at  e 
required. 


The  proper  selection  of  the  computers  to  be  used  for  a  simulation  depends  on 
the  objectives  of  the  study  and  the  nature  of  the  Bystem  being  simulated.  The 
most  appropriate  computer  is  not  necessarily  that  which  has  the  greatest 
sophistication  or  capability.  Most  digital  computers  are  at  a  point  of  develop¬ 
ment  now  which  makes  them  suitable  for  use  in  simulations  (i.e.,  computation 
speed  has  been  increased  to  a  point  where  it  is  suitable  for  solution  of  at  least 
the  low-frequency  system  dynamics).  The  high  accuracy,  exact  repeatability, 
and  flexibility  of  the  digital  make  it  the  most  appropriate  choice  for  overall 
simulation  control  and  for  simulation  of  the  subsystems  which  are  nonlinear  in 
nature.  The  analog  computer,  although  not  as  accurate  as  the  digital,  provides 
continuous  solutions  and  is  more  appropriate  for  solving  higher  frequency 
system  dynamics.  It  is  also  more  appropriate  for  simulating  characteristics 
of  system  hardware  which  operates  in  a  continuous  fashion.  Other  advantages 
of  the  analog  for  simulation  of  vehicle  dynamics  are  more  operational  in 
nature..  The  analog  can  be  programmed  to  represent  linear  vehicle  dynamics 
more  easily  and  debugged  more  quickly  than  the  digital  computer.  Also,  it 
can  be  charged  on-line  without  going  through  the  somewhat  time-consuming 
processes  of  making  card  changes,  recompiling  the  deck  of  cards,  and  reload¬ 
ing  the  program  as  required  for  digital  programs.  Because  of  the  unique 
characteristics  of  tne  analog  and  digital  computers,  a  combination  of  these 
two  computer  types,  or,  a  "hybrid"  computer  system,  is  extremely' attractive 
for  the  simulation  of  real-time,  man-oriented  systems  involving  vehicle 
dynamics. 

The  extensive  use  of  computer  simulation  techniques  by  aircraft  and  space¬ 
craft  systems  engineers  has  made  the  computer  an  integral  part  of  the  systems 
design  and  analysis  process.  This  paper  explains  how  and  where  the  computer, 
specifically  the  hybrid  computer  system,  fits  into  the  design  and  analysis 
process  of  man-oriented  systems  involving  vehicle  dynamics.  Although  the 
discussion  will  be  based  on  this  specific  class  of  systems,  most  of  the  tech¬ 
niques  and  methodology  discussed  would  be  applicable  to  evaluation  of  a  wide 
variety  of  complex  systems.  1 


610 


Manual  IFH  Formation  Flight  System  Model 


The  specific  system  which  will  be  provided  as  an  example  is  a  manual  IFR 
formation  flight  system.  Thi3  system  is  being  investigated  by  Honeywell  under 
the  Joint  Army-Navy  Aircraft  Instrumentation  Research  (JANAIR)  Program, 
a  research  and  exploratory  development  program  to  define  and  validate  advanced 
concepts  which  may  be  applied  to  future,  improved  Naval  and  Army  aircraft 
instrumentation  systems.  This  system  would  allow  a  pilot  to  fly  at  a  fixed 
position  (fixed  range  and  bearing)  with  respect  to  another  aircraft  under  poor 
visibility  conditions.  The  primary  elements  of  the  envisioned  system  (Fig¬ 
ure  2a)  would  be:  1)  an  on-board  special-purpose  computer  to  calculate  infor¬ 
mation  regarding  aircraft  position  in  the  required  f.  rmat  for  display  presenta¬ 
tion;  2)  a  display  which,  will  present  the  necessary  information  for  position 
control  to  the  pilot;  3)  the  pilot,  who  must  make  control  inputs  based  on  the 
information  displayed  to  him;  4)  the  controls  of  the  vehicle,  which  will  move 
the  vehicle  control  surfaces  as  dictated  by  the  pilot's  control  movements; 

5)  the  vehicle  dynamics,  which  will  result  in  rotational  and  translational  move¬ 
ments;  6)  attitude  and/or  rate  sensors  on-board  the  vehicle  which  will  sense 
the  rotational  dynamics  of  the  vehicle;  7)  an  autopilot,  which  will  provide 
feedback  into  the  pilot's  controls  and/or  the  vehicle  dynamics;  8)  navigation 
sensors,  which  will  sense  the  aircraft's  position  with  respect  to  the  lead 
aircraft;  9)  and  a  filter  which  will  smooth  the  data  obtained  from  the  sensor. 

As  mentioned  previously,  the  simulation  of  the  system  must  include  relevant 
environmental  conditions;  two  of  these  are  shown  in  Figure  2a-  i.  e.,  system 
disturbances,  such  as  turbulent  wind  conditions  (10),  and  the  leader  flight 
profiles  (11). 

Figure  2b  shows  a  block  diagram  of  the  organization  of  Honeywell's  hybrid 
system  in  the  mechanization  of  the  formation  flight  system  model.  Honeywell's 
hybrid  system  includes  a  high-speed  digital  computer  with  a  real-time  capability, 
a  number  of  analogs,  a  hybrid  link  for  two-way  communication  between  the 
digital  and  analog,  a  cathode -ray-tute  which  can  be  used  for  generation  of 

"\ 
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•  Figure  2a.  Manual  IFR  Formation  Flight  System  Model 
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Figure  2b.  Simulated  IFR  FF  System 
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desired  display  loruidib  \as  pru^i  duniicu  on  me  ui^mn/i  a  a iTi julatcd  iuotru 
ment  panel  with  a  number  of  conventional  flight  instruments,  a  fixed-base 
pilot  control  station,  and  standard  peripheral  equipment  which  allows  communi¬ 
cation  between  the  computer  and  the  user.  Figure  3  shows  the  control  station 
located  in  front  of  the  cathode  ray  tube  display  and  Figure  4  shows  the  control 
station  and  the  simulated  aircraft  instrument  panel. 

In  the  investigations  of  this  simulated  formation  flight  system,  pilots  have 
been  included  in  the  simulation  loop.  The  flight  control  and  management  task 
which  must  be  performed  by  the  human  operator  consists  of  multi -axis  air-  / 

craft  control  and  three-axis  control  of  position  with  respect  to  another  air¬ 
craft.  This  task  is  much  too  complex  to  be  represented  by  existing  math  models 
of  human  behavior,  which  are  generally  used  to  represent  only  single-axis 
control  tasks. 

When  simulation  runs  are  made,  the  pilot  is  seated  at  the  control  station  in 
front  of  a  display  which  provides  him  with  information  about  his  position  in 
space  with  respect  to  the  lead  aircraft.  The  display  can  also  provide  him 
with  aircraft  control  commands,  in  which  case  the  position  control  task 
becomes  primarily  one  of  three -axis  tracking.  The  flight  profile  of  the  lead 
aircraft  (programmed  on  the  digital  computer)  results  in  the  sequential  per¬ 
formance  of  a  number  of  basic  maneuvers.  The  pilot’s  task  is  to  maintain 
his  position  with  respect  to  the  lead  aircraft  throughout  these  maneuvers. 

Since  this  system  is  to  be  used  for  low  visibility  conditions,  the  pilot  is  pro¬ 
vided  with  no  visual  cues  other  than  the  CRT  display  or  specific  flight  instru¬ 
ments  in  the  simulated  cockpit. 


System  Analysis  and  Design  Process 

The  process  of  system  analysis  and  design  by  simulation  and  empirical  evalua¬ 
tion  can  be  broken  down  into  the  following  tasks:  1)  development  of  the  system 
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Figure  4.  Pilot’s  Station  in  Cockpit  Mockup  with  Electro 
Mechanical  Flight  Instrument  Display 


model;  2)  development  of  the  exper  imental  plan  for  systematic  investigation  of 
system  performance;  3)  development  of  the  computer  simulation;  4)  prelimi¬ 
nary  simulation  runs  to  optimize  the  simulation;  5)  conduct  of  man -in -the -loop 
simulations  according  to  the  prescribed  experimental  plan;  6)  analysis  and 
interpretation  of  system  performance  data;  7)  system  recommendations  and 
formulation  of  conclusions.  As  noted  in  Figure  5.  these  tasks  are  interdepend¬ 
ent,  with  both  the  sequences  and  relationships  between  them  being  important 
factors  in  the  effective  evaluation  of  system  performance. 

One  of  the  mistakes  commonly  made  in  the  use  of  simulation  techniques  is  to 
begin  the  programming  of  the  computer  prior  to  developing  the  math  models 
of  the  system  and  the  experimental  plan  to  be  followed  in  the  evaluation  of 
this  system.  Efficient  organization  of  the  program  requires  at  least  90  per¬ 
cent  completion  of  both  these  tasks  prior  to  development  of  the  simulation 
programs. 

Development  of  System  Model 

The  definition  of  the  system  to  be  simulated  in  terms  of  math  modelB  is  pro¬ 
bably  the  most  difficult  task  in  the  system  design  and  analysis  process.  The 
difficulty  of  this  task  varies  with  the  number  of  subsystems  of  the  model  which 
have  not  been  previously  defined.  If  the  study  is  a  design  problem  rather  than 
an  evaluation  of  an  existing  system,  the  number  of  such  undefined  system 
variables  is  usually  greater.  However,  even  when  subsystems  are  previously 
defined  in  terms  of  hardware  characteristics,  there  are  not  necessarily  exist¬ 
ing  math  models  to  describe  them.  Thus  the  development  of  a  number  of  math 
models  is  usually  necessary  for  any  system  analysis  problem. 

The  complexity  of  the  simulation  to  be  developed  depends  on  the  number  of 
math  models  required  to  define  the  system  adequately  and  the  extent  of  detail 
necessary  for  each  model.  The  validity  of  the  simulation,  of  course,  depends 
on  the  level  of  complexity  selected  for  the  modeling  of  the  system.  Determin¬ 
ing  the  exact  level  of  complexity  suitable  for  a  given  study  is  one  of  the  most 
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Figure  5.  System  Analyses  and  Design  Process  Using  Computer 
Simulation  for  Performance  Evaluation 
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simulation  validity  and  the  scope  of  the  study.  Determining  exactly  what  this 
tradeoff  should  be  requires  a  combination  of  good  judgement  and  a  great  deal 
of  previous  simulation  experience.  Unfortunately,  the  simulation  model 
approaches  exact  duplication  of  the  real  world,  the  cost  of  simulation  analysis 
also  approaches  the  cost  of  flight  testing  the  actual  system. 

The  subsystems  which  have  been  defined  for  the  manual  formation  flight  sys¬ 
tem  model  are  those  shown  in  Figure  2a  (subsystems  numbered  1-11).  Since 
the  primary  objective  of  this  research  program  has  been  to  develop  appropri¬ 
ate  cockpit  displays  for  manual  IFR  formation  flight  with  existing  aircraft,  the 
models  of  the  vehicle,  the  autopilot,  and  vehicle  controls  have  been  predefined 
at  the  outset  of  the  investigations.  With  a  human  operator  as  a  part,  of  the 
simulated  system,  of  course,  it  has  not  been  necessary  to  develop  a  math 
model  for  the  pilot.  The  remaining  subsystem  models  have  been  developed 
during  the  course  of  this  research  program.  The  greatest  amount  of  time  and 
effort  has  been  devoted  to  the  development  of  alternative  display  models. 
Development  of  the  display  model  consists  of  the  following  tasks: 

1)  Identifying  the  information  required  by  the  pilot  to  perform 
the  manual  IFR  formation  flight  control  task. 

2)  Incorporating  this  required  Information  into  a  total  display 
configuration,  which  requires: 

a)  development  of  display  format  (i.e. ,  display  symbology) 
of  the  primary  display, 

b)  development  of  display  driving  functions  (i.e.,  those 
equations  used  to  control  motion  of  display  elements), 

c)  selection  of  standard  .cockpit  instruments  which  should  be 
included  to  provide  the  required  information  which  could 
not  lie  incorporated  into  the  primary  display. 
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Two  of  the  display  configurations  investigated  for  use  in  this  system  are 
shown  in  Figures  6  and  7.  It  may  be  noted  that  both  display  configurations 
shown  provide  the  pilot  with  tracking  symbols  which  indicate  how  he  should 
manipulate  the  controls  to  achieve  the  desired  positions.  The  multi -axis 
control  task  required  for  a  high-order  control  system  such  as  this  becomes 
prohibitively  difficult  if  the  pilot  must  wait  for  the  system's  response  before 
receiving  feedback  regarding  the  accuracy  of  his  control  'nput.  For  example( 
suppose  that  a  pilot  makes  a  roll  input  to  correct  a  lateral  position  error.  His 
control  input  will  instantaneously  effect  a  control  surface  deflection,  producing 
a  roll  rate,  which  in  turn  will  result  in  a  significant  change  in  roll  attitude 
within  a  second  or  two,  which  will  produce  a  trading  change  and  lateral  rate 
of  movement  of  the  aircraft,  which  finally,  after  several  seconds,  will  pro¬ 
duce  a  significant  change  in  the  aircraft's  lateral  position  with  respect  to  the 
leader.  In  other  words,  the  input  response  must  pass  through  several  inte¬ 
grations  before  the  system  finally  responds  with  a  change  in  aircraft  position. 

•1'i 

To  provide  immediate  knowledge  of  the  results  of  his  control  actions,  lead 
information  which  is  based  on  anticipatory  knowledge  of  the  system  response 
must  be  presented  to  the  pilot.  One  means  of  providing  this  information  which 
has  been  used  successfully  in  the  investigations  of  this  manual  IFR  formation 
flight  system  is  display  "quickening".  Quickening  refers  to  the  display  of 
higher-order  deriviatives  of  the  system  response,  which  in  this  case  would 
be  time  derivatives  of  the  follower  aircraft's  position  errors.  Complete 
quickening,  (i.e. ,  presenting  the  sum  of  the  position  error  and  its  derivatives 
in  one  element  on  the  display),  was  utilized  to  drive  the  tracking  symbols 
shown  in  the  display  formats. 


Use  of  Computer  in  System  Model  Development 

The  computer  can  be  used  to  aid  in  developing  the  system  model.  Either  the 
digital,  the  analog,  or  both  can  be  used  for  the  configuration  of  a  simplified 
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Figure  7.  Vertical  Situation  Formation  Flight  Display  (Used  for 
the  Advanced  Rotary- Wing  and  Jet  Fighter  Study) 
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model  of  the  system  to  empirically  evaluate  preliminary  designs  of  subsystems. 
For  example,  in  developing  appropriate  display  driving  functions  for  the  display- 
in  the  manual  formation  flight  system,  performance  of  a  simplified  system 
model  can  be  observed  to  determine  which  combinations  of  feedback  terms  are 
more  appropriate.  This  simplified  model  could  consist  of  only  one  axis  of  the 
vehicle  control  dynamics,  a  simple  model  of  human  behavior  (perflaps  only  a 
lag)  in  response  to  the  specified  display  driving  function,  and  a  continuous 
(analog)  recording  of  vehicle  response  in  terms  of  position  and  attitude.  Small 
programs  can  be  written  to  analytically  evaluate  specific  subsystem  math 
models.  For  example,  to  determine  the  relative  velocity  required  by  the  fol¬ 
lower  aircraft  to  maintain  his  position  with  respect  to  the  leader  during  a  turn 
at  various  ranges  and  bearings,  the  mathematical  relationships  defining  this 
required  velocity  are  quickly  programmed  with  the  ranges  and  bearings  being 
the  variables  in  the  model.  Then  by  merely  typing  in  the  desired  ranges  and 
bearings,  the  corresponding  relative  velocities  canne_^alculat«4^in  a  fraction 
of  a  second. 

In  addition  to  the  incorporation  of  the  system  model,  the  digital  computer 
programs  must  include  the  capabilities  of  overall  simulation  control  arid  com¬ 
puter/user  communication  through  appropriate  input  and  output  channels.  The 
programs  should  be  organized  so  that  it  is  easy  for  the  user  to  change  the 
values  of  any  of  the  system  parameters  which  are  independenj  variables  in  the 
study,  They  should  calculate,  record,  and  output  the  desired  performance 
measures  and  be  structured  to  correspond  to  the  experimental  procedure  to 
be  followed  during  the  course  of  the  man-in-the-loop  simulation  runs.  These 
additional  functions  of  the  computer  simulation  programs  make  it  necessary 
that  the  experimental  plan  for  the  study  be  developed  prior  to  simulation  devel¬ 
opment. 
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Development  of  Experimental  Plan 


The  analysis  of  system  performance  by  simulation,  especially  when  the 
human  Operator  is  included  in  the  simulated  system,  can  involve  many  hours 
of  simulator  runs.  If  too  little  time  and  effort  is  devoted  to  planning  the 
experiffients,  the  results  of  all  these  hours  of  simulator  runs  may  turn  out  to 
be  completely  meaningless.  No  matter  how  complex  or  valid  the  simulation, 
its  usefulness  as  a  tool  in  evaluating  system  performance  depends  on  the 
experimental  plan  followed  in  collecting  the  performance  data. 

When  human  performance  is  one  of  the  contributors  to  system  performance, 
the  system  evaluation  is  based  primarily  on  statistical  inference  from  the 
performance  data  collected.  The  statistical  techniques  which  can  be  used 
in  interpreting  the  performance  data  are  necessarily  limited  by  the  experi¬ 
mental  designs  and  procedures  which  have  been  followed  and  the  exact  per¬ 
formance  measures  which  have  been  recorded.  When  developing  the 
*  '■ 
experimental  plan,  therefore,  it  is  necessary  to  select  the  desired  statistical 

analysis  techniques  and  system  performance  measures,  as  well  as  the 
experimental  design  and  procedure.  The  tasks  involved  and  some  of  the 
factors  which  should  be  considered  in  developing  the  experimental  plan  are 
shown  in  Table  1.  Specific  measures  used  to  describe  performance  of  the 
manual  formation  flight  system  model  are  shown  in  Table  2. 

Computer  Program  Development 

After  the  system  model  and  the  experimental  plan  are  well  defined,  it  is 
possible  to  develop  the  computer  simulation  programs  in  an  efficient  manner. 
I'he  digital  computer  programs  are  written  to  incorporate  all  those  subsys¬ 
tem  models  not  programmed  on  the  analog,  to  calculate  and  record  perfor¬ 
mance  measures,  and  to  allow  effective  communication  between  the  user 
and  the  computer.  It  should  be  organized  in  accordance  with  the  planned 
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Tabic  t 


Development  of  Experimental  Plan 


Identify  Independent  Variables  (IV) 

•  Will  depend  on  study  objectives 

•  Number  of  IV's  limited  by  time  and  money  available 

•  In  man-oriented  system,  human  element  will  be  an  IV 

Identify  Dependent  Variables  (DV) 

! 

•  Complex  system  performance  evaluation  usually  re¬ 
quires  a  number  of  different  performance  measures 

•  When  the  computer  is  used  to  record,  calculate,  and 
output  performance  measures,  additional  costs  asso¬ 
ciated  with  additional  data  are  minimal 

•  Desired  statistical  analysis  techniques  should  be 
considered, 

Identify  Conditions  Which  are  to  Remain  Constant 

•  Will  include  all  subsystems  of  the  Simulated  system 

which  are  not  independent  variables 

/" 

/  ; 

Determine  most  Appropriate  Experimental  design 

•  Limited  by  scope  of  study 

•  Should  be  based  on  study  objectives  and  nature  of 
the  selected  IV's 

Determine  Experimental  Procedure  to  be  Followed 

•  Should  minimize  learning  and  order  effects 

•  Should  maximize  control  over  experimental  constants 

Determine  Statistical  Techniques  to  be  Utilized 

•  Limited  by  experimental  design  and  performance 
measures  selected 

•  Should  relate  to  practical  application  of  study  results 
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Tabic  2.  Performance  Measures  Used  ip  Analysis  of  the 
Manual  Formation  Flight  System 


••  Mean,  standard  deviation,  and  RMS  (root* mean- square) 
error  in: 


•  Longitudinal  position  wrt  leader 

•  Lateral  position  wrt  leader 


•  Vertical  position  wrt  leader 


Range  frOm  leader 


These  measures  are  used  to  indicate  the  pilot's  level  of 
position  control  precision. 


••  RMS  control  stick  rates  (e.  g. , 


X 


i 


where  Xj  is 


rate  of  control  stick  deflection  in  inches  or  degrees  per 
second).  This  measure  is  used  as  an  indication  of  the 
extent  of  pilot  control  activity. 


••  Collisions  with  other  aircraft 


••  Continuous  time  histories  of  position  error,  aircraft 
attitude,  and  control  inputs 

••  Proportion  of  time  subject  utilizes  a  given  display 
••  Number  of  attention  shifts  between  displays 
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as  possible  and  to  free  the  experimenter  from  trivial  tasks  such  aB  timing 
or  data  recording.  It  should  be  organized  such  that  changing  values  of 
system  variables  can  be  accomplished  easily  and  quickly. 

The  analog  computer  is  wired  to  incorporate  those  subsystem  models  which 
can  be  more  conveniently  programmed  and  debugged  on  the  analog  (such  as 
rotational  vehicle  dynamics)  and  the  control  station  is  configured  to  repre¬ 
sent  the  controls  of  the  vehicle  being  simulated. 

Once  the  simulation  is  developed,  it  can  be  used  over  and  over  again  for  a 
long  period  of  time.  It  should,  therefore,  be  well-organized  and  documented 
so  that  it  can  be  used  and/or  modified  in  the  future  with  minimal  difficulty 
The  digital  computer  programs  should  be  modular  in  design  for  ease  of 
checkout  and  modification.  The  wiring  diagram  of  the  analog  simulation 
should  be  detailed  and  complete.  The  program  development  should  not  be 
considered  complete  until  the  program  is  well  documented. 

After  the  entire  system  model  is  mechanized  in  the  form  of  computer  pro¬ 
grams,  the  simulation  is  operational  and  simulator  runs  can  be  conducted, 


Preliminary  Simulations  -  System  Optimization 

Preliminary  simulation  runs  are  required  to  optimize  all  subsystem  models 
based  on  man-in-the-loop  system  performance,  to  optimize  simulation  pro¬ 
cedures,  and  to  experiment  with  the  ranges  of  the  independent  variables  of 
the  study.  It  is  usually  only  after  this  phase  of  the  systems  analysis  process 
that  the  system  model  is  totally  defined.  Although  the  basic  math  models 
for  the  system  must  be  developed  prior  to  writing  the  computer  programs, 
it  is  usually  desirable  to  experiment  with  the  parameters  which  define  the 
exact  characteristics  of  a  given  model  after  the  simulation  is  operable. 

For  example,  the  model  of  a  digital  filter  may  be  specifically  characterized 
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optimal  values  of  these  parameters  are  dependent  on  the  specific  system 
application.  For  complex,  man-oriented  systems  it  is  usually  easier  to 
determine  the  most  appropriate  values  for  such  subsystem  parameters 
empirically  by  means  of  preliminary  simulator  runs. 

When  system  response  depends  on  the  performance  of  a  human  operator, 
there  is  often  no  analytical  way  to  accurately  estimate  the  effect  of  specific 
system  designs  or  parameter  variation  on  total  system  performance.  When 
this  is  the  case,  the  only  way  to  determine  whether  a  given  system  model  is 
functional  is  to  test  it  empirically  with  a  human  operator  in  the  simulation 
loop. 

The  preliminary  simulation  phase  of  the  study  provides  the  analyst  with 
rough  estimates  of  system  performance.  As  a  result  of  this  preliminary 
investigation,  he  can  select  optimal  values  for  parameters  of  the  subsystem 
models,  select  reasonable  ranges  for  the  independent  variables  of  the  study, 
and  redesign  parts  of  the  system  if  necessary.  For  example,  in  the  investi¬ 
gations  of  the  manual  formation  flight  system,  it  was  found  that  the  position 
control  task  was  unreasonably  difficult  when  the  display  provided  only  posi¬ 
tion  error  information.  The  analyst  experimented  with  various  feedback 
terms  and  found  that  the  addition  of  position  error  rate  and  aircraft  attitude 
terms  to  the  position  error  information  made  the  task  much  easier.  Such 
preliminary  investigations  prove  to  be  very  valuable  in  maximizing  the 
effectiveness  of  the  formal  experimental  tests  conducted  in  the  next  phase  of 
the  system  analysis  process. 


Systematic  Man-in-the-Loop  Simulation  for  Collection  of 
Performance  Data _ 


After  the  computer  simulation  has  been  completely  developed  and  optimized 
through  preliminary  simulator  runs,  it  is  finally  in  the  appropriate  stage  to 
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begin  the  systematic  tests  required  for  performance  evaluation.  Althoueh 
conducting  real-time  man-in-the-loop  simulator  runs  may  be  the  most  time- 
consuming  portion  of  the  system  evaluation,  if  preceding  phases  of  the  sys¬ 
tem  analysis  process  have  been  conducted  with  care,  it  is  also  generally  the 
most  straight-forward  part  of  the  study  from  the  system  analyst's  point  of 
view.  This  phase  of  the  system  analysis  process  consists  of  familiarizing 
the  subjects  with  their  task  under  the  various  experimental  conditions  and 
then  conducting  the  simulated  runs  as  prescribed  by  the  experimental  plan. 
For  example,  in  the  investigations  of  the  manual  formation  flight  system,  the 
pilot-subjects  "fly"  the  simulated  aircraft  through  the  programmed  mission 
repeatedly  under  each  of  the  experimental  conditons  until  very  little  further 
improvement  is  noted  in  their  position  control  performance.  Then  each 
subject  "flies"  the  required  number  of  missions  for  the  various  experimental 
treatments  in  the  exact  sequence  prescribed  for  him  by  the  experimental  plan. 
System  performance  data  are  recorded  and  output  for  each  simulator  run. 

The  most  Important  points  to  remember  in  this  usually  extensive  and  repeti¬ 
tive  process  of  system  testing  is  that  the  value  of  the  results  depends  on 
strict  adherence  to  the  prescribed  experimental  plan  and  frequent  checks  on 
the  accuracy  of  the  simulation  programs.  The  experimenter  must  guard 
against  becoming  lax  in  his  experimental  procedure  and  Bhould  frequently, 
preferably  before  each  simulator  run,  perform  a  diagnostic  check  on  the 
simulation  to  assure  that  the  programs  are  working  correctly.,  Although 
exact  repeatability  is  a  characteristic  of  the  digital  computer,  it  Is  not  so 
for  the  analog  or  the  link  between  the  digital  and  the  analog.  The  analog 
and  digital  may  be  improperly  connected,  some  of  the  potentiometers  may 
be  set  incorrectly,  there  may  be  an  amplifier  or  integrator  which  is  not 
plugged  in  securely  or  not  working  correctly  because  of  some  defect,  etc. 

A  diagnostic  check  can  be  performed  which  will  usually  identify  these  pro¬ 
blem  areas  quite  quickly. 


627 


After  all  prescribed  simulator  runs  have  been  completed,  the  system  per¬ 
formance  data  is  sorted  and  collated  for  interpretation  and  analysis  in  the 
next  phase  of  the  system  evaluation. 


Statistical  Analysis  and  Interpretation  of  Performance  Data 

There  are  numerous  statistical  techniques  which  can  be  employed  in 
analyzing  the  performance  data.  The  most  appropriate  techniques  depend, 
of  course,  on  the  study  objectives  and  the  nature  of  the  system  being 
evaluated.  As  previously  emphasized,  the  techniques  to  be  used  should  be 
selected  when  the  experimental  plan  and  procedure  are  being  defined,  to 
assure  that  the  performance  measures  recorded  and  the  plan  followed  will 
allow  valid  application  of  the  desired  techniques. 

The  digital  computer  can  be  used  to  perform  the  lengthy  calculations  required 
for  the  statistical  tests.  Most  large  digital  computer  facilities  have  a  num-  ' 
ber  of  general- purpose  statistical  programs  available  for  performing  the 
more  commonly  used  and  well-known  statistical  tests.  A  number  of  such 
programs  are  available  at  Honeywell  and  have  been  used  in  analyzing  data 
obtained  in  tests  of  the  formation  flight  system. 

When  the  computer  is  used  both  for  calculating  and  recording  data  and  for 
performing  the  statistical  tests,  the  added  expense  associated  with  recording 
additional  performance  measures  and/or  performing  a  number  of  different 
tests  on  this  data  is  relatively  low.  Since  it  is  often  difficult  for  the  analyst 

to  determine  which  performance  measures  are  more  appropriate  until  .after . 

the  data  has  been  collected  and  analyzed.  It  is  a  good  idea  for  him  to  record 
all  those  measures  which  seem  to  be  relevant.  If  digital  computer  programs 
are  available  for  performing  the  desired  statistical  tests,  the  same  philosophy 
can  be  followed  in  determining  the  number  of  tests  to  be  applied  to  the  data. 
Kvcn  when  the  scope  of  the  study  limits  the  statistical  analysis  effort,  it  is 
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important  that  sufficient  performance  data  be  collected  and  saved.  It  is  often 
desirable  to  perform  further  tests  later  as  additional  funding  becomes  avail¬ 
able.  Also,  it  may  be  that  a  future  study  with  different  objectives  requires 
different  methods  of  data  analysis  but  could  be  based  on  the  same  system 
performance  data.  Some  of  the  statistical  techniques  which  have  been  used 
in  evaluating  the  manual  formation  flight  system  are  shown  in  Table  3. 

After  results  of  the  ste'.uuical  tests  are  obtained,  they  must  be  interpreted 
in  terms  of  their  implications  in  designing  or  developing  the  system.  Some¬ 
times  results  which  arc  statistically  significant  are  not  significant  from  a 
practical  systems  application  standpoint.  For  example,  suppose  that  the 
results  of  a  statistical  comparison  of  two  displays  for  the  manual  formation 
flight  system  showed  that  lateral  position  control  performance  was  signifi¬ 
cantly  better  for  one  display,  and  that  the  average  position  errors  for  this 
display  were  consistently  five  feet  lower  than  those  for  the  other  display. 

No  matter  what  the  level  of  statistical  significance,  this  small  difference  in 
position  error  may  hot  be  of  practical  significance  in  terms  of  system 
development.  If  performance  resulting  from  two  such  displays  were  this 
similar,  the  system  analyst  would  probably  recommend  that  display  which 
■would  be  less  expensive  to  incorporate  into  the  system.  Since  the  only 
results  of  real  significance  to  the  system  analyst  are  those  which  can  be 
related  to  the  design  or  development  of  the  system,  it  is  important  that 
sufficient  time  be  devoted  to  interpreting  the  results  of  the  statistical  tests 
accordingly. 

After  the  analyst  has  decided  what  the  important  Btudy  results  are,  these 
results  should  be  presented  clearly  in  the  final  study  report.  Since  the 
graphical  form  of  presenting  data  is  usually  more  easily  intcrpretable  for 
the  reader  than  the  tabular,  it  may  be  helpful  to  sltow  at  least  the  more 
important  results  graphically. 
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Table  3.  Statistical  Tests  Used  to  Evaluate  Performance  of 
Manual  IFR  Formation  Flight  System 


Statistical  Procedures: 

Based  on  Following  Performance 
Measures: 

Factorial  analyses  of  variance 

1)  RMS  position  error 

2)  RMS  control  stick  rate 

3)  Mean  position  error 

Calculation  of  mean  and/or  median 
position  errors  for  factorial  com¬ 
binations  of  independent  experi¬ 
mental  variables. 

Mean  position  error  for  a  specific 
subject,  treatment,  and  maneuver. 

Calculation  of  mean  and/o.'  median 
variances  for  factorial  coir  bination  s 
of  independent  experimental 
variables. 

Standard  deviations  around  the  mean 
position  error  for  a  specific  subject, 
treatment,  and  maneuver. 

Calculation  of  correlation  and/or 
regressions  between  system 
variables. 

E.  g. .  between  lateral  and  longitu¬ 
dinal  position  errors,  or  between 
fore-aft  and  right-left  control  stick 
movements,  etc. 
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formulation  o!  Conclusions  and  System  Recommendations 


The  final  stop  in  the  system  evaluation  process  is  the  careful  examination 
of  study  results  to  formulate  conclusions  and  make  system  recommendations. 
Sometimes  in  the*  early  phases  of  the  system  investigation,  recommendations 
regarding  system  implementation- -such  as  required  hardware  character- 
istics--cannot  be  made  until  further  research  har  been  conducted.  The 
study  conclusions  at  this  point  uauully  relate  primarily  to  the  feasibility  of 
system  concepts.  The  primary  system  recommendations  made  will  be 
suggested  areas  for  further  investigation,  utilizing  those  effective;  system 
concepts  and  models  which  have  been  developed  during  the  previous  system 
studies  as  a  basis  for  future  studies.  The  system  analysis  process  des¬ 
cribed  in  this  paper  would  then  be  repeatud  a  number  of  times  until  suf¬ 
ficient  system  aspects  have  been  investigated  to  allow  recommendations 
regarding  hardware  characteristics  and  specifications. 

Results  and  conclusions  of  the  formation  flight  syBtem  research  are  pro¬ 
vided  below  as  an  example  of  the  steps  which  must  be  completed  prior  to 
actual  system  development  and  the  type  of  conclusions  which  can  be  drawn 
from  system  evaluation  studies  based  on  computer  simulation  analysis. 

Results  and  Conclusions  of  Research  on  Manual  IFR 
'  Formation  Flight  System _ 


The  research  on  the  formation  flight  system  has  not  yet  reached  the  point 

\ 

where  the  exact  specification!  of  required  hardware  characteristics  would 

i 

bo  desirable.  The  philosophy  followed  in  this  system  investigation  has 
been  to  first  determine  the  feasibility  of  basic  system  concepts  and 
associated  system  performance,  assuming  no  limitations  imposed  by  sys¬ 
tem  hardware.  Then,  one  by  one,  the  system  limitations  imposed  by 
realistic  hardware  characteristics  and  expected  environmental  conditions 
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haw  been  added  and  resultant  system  performance  evaluated.  Since  there 
are  so  many  system  variables  in  a  complex  system  such  as  this,  it  has  not 
been  desirable,  either  from  a  cost  or  operational  standpoint,  to  investigate 
all  these  variables  simultaneously.  Instead,  a  number  of  different  studies 
have  been  conducted  and  each  new  study  has  utilized  results  of  previous 
studies  as  a  basis  for  evaluating  additional  system  variables.  The  studies 
which  have  been  performed  to  investigate  the  manual  IFR  formation  flight 
system  and  the  major  results  of  these  studies  are  described  briefly  below. 

The  first  study  (Reference  1)  of  this  program  was  conducted  to  investigate 
basic  information  requirements  for  the  manual  IFR  formation  flight  control 
task  for  rotary-wing  aircraft  and  to  evaluate  display  concepts  for  a  com- 
puter-addressed  cathode-ray  tube  display.  The  effects  of  turbulent  wind 
conditions  and  subsidiary  pilot  workload  were  also  investigated  in  this  study. 
This  study  assumed  no  limitations  on  sensor  outputs  (1.  e, ,  high  data  trans¬ 
mission  rate  and  no  measurement  noise).  Some  o f  the  study  results  are 
shown  in  Figures  8  through  1 1.  The  major  conclusion  of  the  study  was  that 
formation  flight  under  IFR  conditions  appears  to  be  a  realizable  goal  with 
the  aid  of  the  computer- generated  display  formats  developed. 

The  second  study  (Reference  2)  was  conducted  to  evaluate  an  existing  heli¬ 
copter  formation  flight  system.  This  study  assumed  the  sensor,  computer, 
and  display  characteristics  of  the  system  being  evaluated.  Results  of  this 
study  demonstrated  the  Important  effects  of  filtering  techniques,  data  trans¬ 
mission  rate,  and  display  driving  functions  on  total  system  performance 
(see  Figures  12  through  14). 

A  third  study  (Reference  3)  was  conducted  to  evaluate  the  effectiveness  of 
conventional  flight  instruments  in  a  manual  IFR  helicopter  formation  flight 
system.  Two  state-of-the-art  electro-mechanical  displays,  i.  e. ,  a  flight 
director  and  a  horizontal  situation  indicator,  were  used  in  conjunction  to 
display  the  required  information  and  were  evaluated  under  alternative  display 
formats.  This  study  again  assumed  no  limitations  on  sensor  system  outputs. 
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MEDIAN  RMS  ERROR  (FEED 


THE  GRAPH  BELOW  SUMMARIZES  THE  K' SUITS  OF  THE  PRELIMINARY 
SIMULATIONS  DESIGNED  TO  SELECT  OPTIMIZED  DISPLAY  FORMATS 
FOR  THE  CONVENTIONAL  HELICOPTER. 


A 


DISPLAY  CONFIGURATIONS 
*  Chosen  lor  further  study  end  evaluation 


Figure  8.  Original  Study  of  Concept  Feasibiltiy  and  Display 
Requirements  for  the  Conventional  Helicopter 


DISPLAY  FORMATS 


Figure  9.  Comparative  Evaluation  of  Display  Formats  for 
Conventional  Helicopter  Study 
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MEAN  RMSV  ERROR  <  FEET  I 


Figure  10.  Effects  of  Turbulence  on  System  Performance 
Conventional  Helicopter  Study 


»  '  ! 


Figure  11.  Effects  of  Subsidiary  Pitot  Workload  on  System 
Performance  -  Conventional  Helicopter 
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MEDIAN  RMS  ERROR  (FEET) 


THE  GRAPH  BELOW  SHOWS  MEAN  RMS  ERRORS  PLOTTED  AS  FUNCTIONS 
OF  THE  SYSTEM  UPDATE  TIME  INTERVAL.  SYSTEM  UPDATE  RATE  IS  THE 
RECIPROCAL  OF  SYSTEM  UPDATE  TIME  INTERVAL. 
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Figure  12.  Sample  Prototype  System  Evaluation  -  Effect  of 
System  Update  Rate  on  System  Performance  - 
Conventional  Helicopter 

/ 


Figure  13.  Sample  Prototype  System  Evaluation  -  Effects 
of  Terms  in  Display  Quickening  Equations  on 
System  Performance 
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The  results  of  this  study  (see  Figures  15  and  16)  indicated  that  it  was  possible 
to  obtain  position  control  with  the  electro- mechanical  displays  comparable  to 
that  obtained  with  the  computer-generated  displays. 

The  fourth  study  (Reference  4)  investigated  the  relationship  of  variations  in 
the  effective  data  transmission  rate  (i.  e, ,  defined  as  the  update  rate  of  the 
information  presented  on  the  displays)  and  the  effective  level  of  measurement 
noise  (i.  e, ,  noise  which  appears  on  the  display  after  filtering)  on  pilot/ 
system  performance  in  the  helicopter  IFR  formation  flight  mode.  Results 
(see  Figures  17  through  19)  showed:  1)  that  position  control  performance 
degraded  with  increasing  measurement  noise;  2)  that  position  control  per-  . 
formance  did  not  improve  significantly  by  increasing  the  update  rate  above 
4/ second;  and  3)  that  optimal  display  driving  functions  and  data  filtering 
techniques  are  dependent  on  the  data  update  rate  and  accuracy  characteristics 
of  the  system. 

The  objective  of  the  fifth  study  (Reference  5 )  was  to  evaluate  the  effect  of 
varying  levels  of  automatic  control  assistance  on  pilot/  system  performance 
in  the  simulated  helicopter  IFR  formation  flight  mode.  An  information  update 
rate  of  4/second  and  a  moderate  noise  level  were  assumed  throughout  this 
study.  Results  of  the  study  (see  Figures  20  and  21)  indicated  that  increasing 
the  level  of  automatic  control  assistance  provided  greater  system  stability 
and  made  the  pilot's  control  task  less  demanding,  but  did  not  significantly 
improve  position  control  performance  over  that  obtainable  manually  with  the 
aid  of  the  quickened  display. 

The  sixth  study  (Reference  6)  was  conducted  to  define  information  and 
display  requirements  and  investigate  variable  sensor  output  characteristics 
for  two  additional  vehicle  classes,  i.  e. ,  the  advanced  rotary-wing  and  the 
jet  fighter  aircraft.  The  results  (see  Figures  22  through  25)  indicated  that 
manual  IFR  formation  flight  with  the  envisioned  system  appears  to  be  feasible 
for  the  advanced  rotary-wing  and  the  jet  fighter  aircraft  and  that  the  effects 
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MEAN  RUS  ERRORS  (FEET) 


SINGLE-CUE  DOUBLE-CUE 

Figure  15.  Comparative  Evaluation  of  Two  Flight  Director 
Displays  for  Conventional  Helicopter 
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Figure  1G. 


Computer-Generated  Display  versus  Conventional 
Flight  Director  Display  -  Conventional  Helicopter 


DATA  RATES  (UPOATE  S/SECOND) 

Figure  17,  RMS  Position  Errors  by  Data  Rate  - 
Conventional  Helicopter 


ACCURACY  LEVEL 

Figure  18.  RMS  Position  Errors  by  Data  Measurement  Noise 
Level  -  Conventional  Helicopter 
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19.  Horizontal  Position  Error  Envelope  for  Conventional 
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Figure  22.  Horizontal  Error  Envelope  by  Maneuver  for  the 
Advanced  Rotary-Wing 


Figure  23.  Horizontal  Position  Error  Envelope  by  Maneuver 
for  the  Jet  Fighter 
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Figure  24,  Effects  of  Data  Update  Rate  on  System  Performance 
Jet  Fighter 
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Figure  25.  Effects  of  Sensor  Measurement  Noise  on  System 
Performance  -  Advanced  Rotary-Wing 


on  system  performance  of  variation  in  measurement  noise  and  display 
update  rate  were  the  same  as  found  previously  for  the  helicopter. 

Some  of  the  basic  conclusions  which  have  been  drawn  as  a  result  of  these 
research  studies  are  summarized  below. 

Display  driving  functions  are  more  important  thin  the  display  format  in 
determining  the  pilot's  control  performance.  The  results  of  the  various 
display  evaluations  suggest  that  as  long  as  the  basic  display/ control 
relationships  are  satisfied,  all  required  information  is  presented,  and  the 
display  formats  are  interpretable,  a  number  of  different  display  formats  (in 
terms  of  specific  symbology  and  orientation)  are  appropriate  for  the 
envisioned  1FR  formation  flight  system. 

Manual  IFR  formation  flight  with  the  system  as  modeled  for  these  research 
studies  appears  to  be  a  realisable  goal  for  the  conventional  helicopter,  the 
advanced  rotary-wing,  and  the  jet  fighter.  The  level  of  position- control 
precision  to  be  expected  for  a  specific  aircraft  class,  given  the  presentation 
of  all  required  information,  the  use  of  an  appropriate  display  format,  and 
optimal  display  quickening,  will  be  a  function  of  at  least  the  following 
variables: 

1)  Rate  at  which  new  information  is  available  for  display 

2)  Level  of  measurement  noise  on  positon  information. 

3)  Filtering  technique  used  to  smooth  the  noisy  data 

4)  Experience  level  and  capability  of  the  individual  pilot 

5)  Extent  of  pilot  workload  required  for  subsidiary  taskB. 

It  is  suspected  that  other  variables,  such  as  pilot  fatigue,  aircraft  separa¬ 
tion  distances,  formation  geometry,  formation  airspeed,  command  mode 
(i.  e, ,  fly-to  or  fly-from),  lead  aircraft  perturbation,  etc.,  also  effect 


postion- control  performance.  Current  research  under  the  JANAIR  program 
includes  investigation  of  a  number  of  these  additional  system  variables. 

Prior  to  exact  specification  of  the  hardware  required  for  the  manual  IFR 
formation  flight  system,  these  and  other  seemingly  relevant  system  variables 
should  be  investigated. 


The  Value  of  Simulation  in  System  Performance  Evaluation 

The  conclusions  summarised  above  are  som^iat  tentative  in  nature  and  do 
not  specify  the  exact  level  of  system  performance  which  can  be  expected  with 
the  developed  system  under  actual  flight  conditions.  It  must  be  emphasized 
that  simulation  analysis  can  not  be  a  substitute  for  actual  system' tests. 

Rather,  simulation  is  a  compromise  method  of  evaluating  system  performance 
with  a  degree  of  validity  which  falls  somewhere  between  preliminary  pencil- 
and- paper  analyses  and  tests  of  an  actual  breadboard  system.  Exactly  where 
it  falls  in  terms  of  validity  depends  on  the  complexity  of  the  system  model 
which  is  developed  for  the  simulation. 


The  value  of  simulation  as  a  system  evaluation  tool  is  greatest  for  those 
systems  which  cannot  be  easily  defined  analytically  (such  as  those  involving 
complex  human  behavior)  and. which  cannot  be  tested  under  real-world  condi¬ 
tions  without  extremely  high  risks  (i.e, ,  in  terms  of  human  life  and/or 
system  co!sts).  For  this  type  of  system,  compromise  methods  of  system 
evaluation  are  neceriary,  and  they  are  not  meant  to  replace  actual  system 
tests,  but  to  precede  and  minimize  the  extent  of  these  tests. 


i 

Although  there  are  inherent  constraints  and  limitations  in  the  evaluation  of 
system  performance  by  simulation  analysis,  it  is  extremely  useful  in  answer¬ 
ing  questions  such  as  the  following: 


•  Is  a  given  system  design  or  concept  feasible? 

•  How  does  one  method  or  design  compare  to  another? 

•  What  are  reasonable  minimum  and  maximum  limits  on  the 
variation  of  a  given  system  parameter? 

•  What  are  the  effects  of  varying  two  or  more  system 
parameters  simultaneously? 

•  Can  the  human  operator  perform  the  required  task? 

•  Which  3ystem  tasks  should  be  performed  automatically? 

•  What  is  the  general  relationship  between  system  performance 

and  a  given  eyitem  parameter  or  environmental  condition? 

/ 

I 

If  the  system  analyst  conducts  his  system  investigation  scientifically  and  if 
he  tempers  his  formulation  of  conclusions  and  system  recommendations  by 
acknowledgement,  of/ the  constraints  and  limitations  of  the  simulation,  he 
will  find  computer  simulation  techniques  to  be  an  invaluable  tool  in  the 
system  analysis  and  design  process. 
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r,  mss  ary 


Activity  indices 

A/C 

Autopilot  mode 

Concomitant  task 


Mean-square  pitch  and/or  roll  rates  of  the 
simulated  aircraft 

Aircraft 

Level  of  automatic  control  included  in  the 
aircraft  control  system.  Ranges  from 
simple  damping  in  a  single  axis  of  the  air¬ 
craft  to  control  of  the  aircraft's  heading  and 
altitude. 

A  class  of  secondary  task,  which  is  performed 
simultaneously  with  the  primary  task  and  is 
highly;  quantifiable  in  nature.  In  the  study 
referenced  in  this  paper,  the  primary. air¬ 
craft  position  control  task  was  performed 
continuously  and  the  pilot's  formation  flight 
display  was  Intermittently  blanked  out.  Then 
secondary  task  cues  were  provided,  requiring 
that  th,e  pilot  simultaneously  perform  the 
secondary  task  at  a  forced -pace  (i.e.,  the 
frequency  and  time  interval  of  the  formation 
flight  display  Interruption  was  controlled  by 
the  experimenter  rather  than  left  to  pilot 
discretion).  The  levels  of  the  secondary 
workload  we^e  defined  in  terms  of  percentage 
of  time  the  pilot  was  required  to  perform  the 
secondary  task. 
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Data  transmission  and/or 
update  rate 


Dependent  variables 


Double -cue  flight  director 


Filter 


IFR 


The  rate  at  which  new  information  about  the 
follower  aircraft's  position  in  space  (with 
respect  to  the  lead  aircraft  of  the  formation) 
is  available  for  display. 

Variables  of  an  experiment  which  describe 
system  performance.  These  performance 
measures  are  assumed  to  reflect  changes  in 
the  levels  of  the  independent  variables  of  the 
experiment  and  are  thus  considered  to  be 
"dependent". 

Electro-mechanical  flight  instrument  cur¬ 
rently  used  in  both  fixed-wing  and  helicopter 
aircraft  to  provide . information  about  aircraft 
pitch  and  roll  attitudes.  The  "double -cue" 
flight  director  has  two  separate  moving  ele¬ 
ments,  one  representing  pitch  and  the  other 
roll  attitude  deviations. 

In  die  referenced  studies  the  filter  simulated 
was  a  digital  a-(3  filter.  The  filter  is  required 
to  smooth  the  aircraft  position  information 
obtained  from  the  assumed  sengor  system. 
Sensor  systems,  ape  .usually  characterised  by 
a  certain  level  of  measurement  noise. 

i  • 

Instrument  Flight  Rules  --  This  term  is  used 
in  this  report  to  refer  to  very  low  visibility 
conditions  when  -*».  pilot  would  have  tc  depend 
primarily  or.  instrument*  for  visual  cues. 
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IHAS  display  format 


Independent  variables 


Mean  position  error 


Measurement  noise 


I 


Integrated  Helicopter  Avionics  System  - 
The  vertical  situation  display  format  which 
was  intended  as  a  part  of  this  avionics  system 
was  configured  for  use  in  the  IFR  formation 
flight  mode  in  one  of  the  referenced  studies. 

Parameters  of  a  system  which  are  varied  to 
investigate  their  effect  on  system  perform¬ 
ance. 

N 

X./N  ,  where  X.  was  the  measured 
i-1 

position  in  a  specific  axis  during  a  specific 
maneuver  and  N  was  the  total  number  of  posi¬ 
tion  measurements  recorded  during  the  ma¬ 
neuver.  For  the  referenced  studies  N  was 
equal  to  the  total  maneuver  length  times  the 
display  update  rate. 

Used  to  refer  to  the  error  In  the  measurement 
of  the  follower  aircraft's  position  with  respect 
to  the  leader.  The  sensor  system  was  as;- 
sumed  to  include  noise  with  a  normal 
{Gaussian)  distribution  described  by  the 
standard  deviation.  In  the  referenced  studies 
these  standard  deviations  were  defined  in 
terms  of  the  bearing  (Og),  elevation  (aE),  and 
range  <o^)  measurements  (  in  degrees),  or  in 
terms  of  the  longitudinal  (a^,),  lateral  (ffy), 
and  vertical  { a position  (in  feet). 
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Perspective  display  format 


PPI  display 


Quickening 


RMS 


A  display  format  configured  for  the  formation 
flight  mode  which  provided  the  pilot  with  a 
three-dimensional  representation  (in  two 
dimensions)  of  his  formation  position  with 
respect  to  the  leader. 

Plan  Position  Indicator  -  A  display  format 
configured  for  the  formation  flight  mode 
which  presented  a  horizontal  view  of  all  air¬ 
craft  in  the  formation. 


A  method  of  providing  lead  or  anticipatory 
information  regarding  the  system's  response. 
As  used  in  the  referenced  studies,  it  consisted 
of  adding  higher  order  derivatives  (i.e. ,  rate 
of  change  of  position)  of  the  system's  response 
(change' in  aircraft  position)  to  the  actual  posi¬ 
tion  error.  The  resultant  sum  was  used  to 
drive  one  element  on  the  display. 


Root -Mean  -Square  -  — 


X,  / N  ,  where 


X.  was  the  measured  position  in  a  specific 
axis  during  a  specific  maneuver  and  N  was 
the  total  number  of  position  measurements 
recorded  during  the  maneuver.  The  RMS 
measure  was  also  used  to  represent  the  levels 
of  pilot  and  aircraft  control  activity  during 


a  given  maneuver,  in  which  case  X(  represented 
reapectively  the  rate  of  movement  of  the  con¬ 
trol  stick  or  the  aircraft's  attitude. 
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SAS 


Stability  Augmentation  System. 


Single-cue  flight  director  An  electro -mechanical  flight  display  which 

has  only  one  moving  element  to  represent 
both  pitch  and  roll  attitudes  of  the  aircraft. 

The  element  represents  both  aircraft  axes 
by  movement  in  two  different  axes  on  the 
display. 

SK/FF  Stationkeeping/ Formation  Flight. 

Standard 

value  of  the  observations  (X()  and  N  is  the 
total  number  of  observations. 

Subsidiary  pilot  workload  The  pilot's  workload  on  tasks  other  than  his 

primary  task  (aircraft  position  control  in 
the  referenced  studies).  See  the  description 
of  "concomitant  task"  for  more  detail  on  how 
this  subsidiary  workload  was  simulate^. 

System  update  time  interval  Reciprocal  of  the  rate  at  which  the  pilot  re¬ 
ceived  new  information  about  his  position  with 
respect  to  the  formation  leader  for  display, 

a  Coefficient  of  or-fi  digital  filtering  model  which 

determines  weight  of  current  raw  position 
measurement  versus  average  over  old 
measurements. 

0  Coefficient  of  a-3  digital  filtering  model  which 

determines  weight  of  current  velocity  measure¬ 
ment  versus  average  over  old  measurements. 
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EXPERIMENTAL  DESIGN  CONSIDERATIONS  IN  VALIDATING  A  METHOD 
OF  MODELING  A  MAN-ORGANIZED  SYSTEM 


Bob  B.  Lukens  and  Robert  A.  Brown 
University  of  Alabama 
Huntsville,  Alabama 


ABSTRACT .  During  the  past  decade,  significant  advancements  in 
business  organization  modeling  have  been  achieved.  The  man-organized 
system  is  a  system  whose  elements  consist  of  people,  material,  money, 
and  information.  Various  methods  and  procedures  have  been  developed 
to  model  this  business  organization.  The  purpose  of  the  research 
project  la  to  investigate  the  theoretical  aspects  of  validating  a 
method  of  modeling  a  man-organized  system. 

A  new  approach  to  modeling  the  organization  is  a  method  called 
Dynamic  Organization  Network  Analysis  (DONA).  The  basis  for  this 
method  is  the  use  of  state  variable  equations.  A  question  arises 
concerning  the  validity  of  the  DONA  model.  This  is  answered  through 
an  experimental  system  designed  to  test  the  model.  The  experimental 
system  incorporates  a  computer  simulation  with  known  characteristics. 
The  simulation  is  used  as  a  standard  for  comparing  the  performance 
characteristics  of  the  DONA  model. 


The  remainder  of  this  article  has  been  reproduced  photographically 
from  the  authors'  manuscript. 
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EXPERIMENTAL  DESIGN  CONSIDERATIONS  IN 
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Introduction 

During  the  past  decade,  significant  advancements  in  business  organization 

modeling  have  been  achieved.  Various  procedures  and  methods  have  been  developed 

to  model  systems  including  man-organized  systems.  The  man-organtzed  system  is 

characterized  by  elements  such  as  people,  material,  money,  and  information.  A 

method  widely  used  in  modeling  large  organizations  is  computer  simulation.  One 

of  the  most  extensijve  works  on  computer  simulation  of  large  organizations  Is  by  Jay 

3 

W.  Forrester.  In  his  book  "Industrial  Dynamics"  he  introduces  l.is  concept  of  modeling 
organizations  and  lie  describes  a  new  computer  language/  DYNAMO,  to  Implement 
the  modeling  technique.  Forrester's  method  is  suited  to  a  project  which  requires  modeling 
of  fine  detaili  of  at)  organization.  However,  the  method  may  require  years  to  fully 
model  a  large  organization.  The  need  for  a  method  of  modeling  to  be  accomplished 
in  a  timely  manner  caused  a  search  for  other  approaches. 

Within  the  last  few  years,  state  variables'  have  gained  attention  as  a  tool  to 
be  used  In  systems  analysis. 1 The  state  variable  equations  have  found  use  In 
mechanical,  hydraulic  and  electrical  systems.  Herman  E.  Koenig,  at,  al.  In  the  book 
"Analysis  of  Discrete  Physical  Systems"  proposed  that  the  analysis  may  be  carried  Into 
socio-economic  systems.  The  strength  of  this  approach  lies  in  the  foct  that  the  systems 
is  considered  from  a  control 'System  point  of  view,  The  basic  postulate  is  that  the  system 
is  regarded  as  a  conservative  system.  The  matching  of  inputs  to  outputs  requires  for- 
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mutation  according  to  energy  considerations.  This  approach  differs  from  the  economists 
approach  in  that  the  economist  views  the  system  as  a  mock  box  wii'i  only  Input;  and 
outputs.  Other  quantities  such  as  internal  performance  characteristics  ore  not  given 
detailed  consideration  by  the  economists.  By  using  the  control  theory  point  of  view, 
the  analyist  is  capable  of  manipulating  those  quantities  which  control  the  performance 
of  the  organization. 


The  Dynamic  Organizotlonal  Network 
Analysis  Model 

'  The  state  variable  approach  was  used  In  the  modeling  of  a  socio-economic  system 
5  8' 

'  by  Koenig.  '  Koenig's  procedure  paralleled  Forrester's  In  that  mathematical  relation¬ 
ships  have  to  be  developed  for  all  significant  operations.  This  Is  very  time  consuming 
for  a  large  organization.  The  fact  that  only  two  forms  £f  equations  are  needed  In  the 
state  variable  approach  caused  further  considerations  of  this  approach.  The  state  variable 
equations  are  as  follows: 

T  (t)  ■  P  Yfr)  +  QX(t) 

Y(t)«M  T(t)  +  N  X(t) 

e 

where  P,  Q,  M,  and  N  are  matrices  characteristic  of  the  system,  X(t)  Is  the  vector 
of  inputs,  Y(t)  Is  the  vector  of  outputs,  and  the  'f  (t)  are  state  variables. 

Rewriting  these  equations  in  matrix  multiplication  form  yields: 


This  form  suggests  a  form  appearing  in  multiple  regression  analysis.  With 

these  concepts  in  mind,  a  modeling  method  called  Dynamic  Organisational  Network 

J' 

Analysis  (DONA)  evolved. 

The  DONA  method  of  modeling  is  a  self-generating  procedure  in  that  the 
matrices  characterizing  the  system  are  generated  from  the  system  through  regression 
analysts.  Regression  analysis  requires  data  from  the  system  and  this  data  will  certainly 
consist  of  discrete  quantities  or  measurements  taken  at  some  time  Interval.  Since  the 
state  variable  equations  describe  continuous  functions, „a  discrete  form  may  be  derived 


by  writing  the  equations  in  difference-equation  form. 


V  (t  +  hj 


+  1)  hQl  I*  V(t)“| 

W  N  J  '  |x(oJ 


^  I 


Taking  first  differences  to  high-pass  filter  the  records,  thus  eliminating  trends, 


yields 


.)  1 

= ,  Rp  +i)  Ql  [*<*)■ 

_v(Y)  M  N  *(X) 


where 


MO  «Y(t  +  1)  -  Y(t) 
*  (Y)  -  Y(»)  -  Y(t  -  1) 


for  h  =  I 


Letting  the  matrix  [5-  * 


is  RP  +1)  Q1 

L  M  NJ 


and  substituting  yields 


MO  i  *  CO 

*  S 

v  (Y)  V  (X 
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The  ^X,  ^Y,  ,  and  -V  are  determined  from  calculations  using  actual  data 

produced  by  the  reai  system  to  be  modeled.  The  time  interval  may  be  one  liou> ,  une 
week,  etc.  By  the  use  of  a  computer  program  for  multiple  regression  analysis,  the 
characteristic  matrix,  S,  can  be  determined. 


The  Laboratory  Concept  for  Validation 


A  question  arises  concerning  the  validity  of  the  DONA  model  and  the  method 
which  produced  the  model.  The  usual  procedure  for  validating  such  a  model  is  by  using 


information  from  the  real  system  and  determining  if  the  model  predicts  in  an  acceptable 
manner -when  compaKd  with  the  performance  of  the  real*  system.  It  was  felt  that  a 
better  method  could  be  usechto  validate  the  DONA  model.  A  laboratory  concept  was 
developed  to  validate  the  method,  which  In  turn  validates  the  model  produced  by  the 
method.  In  this  concept  the  pdramaters  can  be  controlled  to  determine  the  range  and 

responsiveness  of  the  model.  ,The  experimental  system  shown  in  Figure  1  is  used  for  the 

'  /  •  •  I 

validation  procedure.  The  block  marked  "SIMCO"  is  a  computer  simulation  of  a  sales 
conippny.  This  slmulatlorf^was  developed  by  C.;McMJllan  and  Richard  F.  Gonzales. 

As  orglnal I y  written  SIMCO  was  a  distributor  operation  for  a  single  product.  Stochastic 
deitiand  and  lead  Hiries  were  Incorporated,  SIMCO  was  modified  to  handle  a  second 
product  and  to  simulate  personnel  actions;  i.e.  hires,  fires,  and  transfers.  These  modi- 
fications  were  made  to  widen  the  scope  of  operations  through  the  addition  of  the  second 
product  and  to  have  some  interaction  of  elements;  e.g.  the  transfer  of  personnel  from 
one  product  line  to  the  other.  All  of  the  characteristics  of  SIMCO  ore  known.  Since 


SIMCO  is  a  subroutine,  it  can  easily  be  replaced  to  study  the  validity  of  DONA  as 
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applied  to. any  other  simulated  activity. 


Tk<i  N  j  A  m  l*gy.  U  !*«•!#  |m  p  |  m  1  ill*  wiij  J  i  |p  I  *  #*  g  |  ** 

analysis  procedure  for  developing  the  "3"  matrix.  In  the  laboratory  system  the  "S" 
matrix  will  characterize  the  SlMCO  Sales  Company.  The  DONA  model  block  in  Figure 
I  represents  the  following  matrix  equation. 


Y(t  +  1) 

■>  m 

i  (0 

as 

3 

• 

1 _ 

m  m 

.X(t). 

The  left  side  of  the  matrix  equation  i»  the  DONA  output., 

The  outputs  of  SIMCO  and  the  DONA  model  art  finally  compared  as  shown  In 
Figure  1 .  The  comparison  is  made  on  all  parameters  desired  or  deemed  ppproprlate  for 
consideration.  For  example,  the  DONA  model  not  only  produces  the  system  outputs 
but  also  predicts  for  the  next  time  interval  the  value  of  the  state  variables. 

•  r 

Experimental  Design  Considerations  in  the  Validation  Procedure 

The  problems  revealed  In  this  project  have  provided  some  valuable  Insights 
Into  modeling  of  a  man-organized  system  using  this  method.  The  laboratory  concept 
described  above  has  been  fully  implemented.  The  entire  concept  has  been  written  into 
c  computer  progrom  and  the  progrom  has  been  run  and  debugged. 

Since  thi-  is  a  laboratory,  the  generation  of  stimulus  data  was  the  first  big 
problem.  Even  though  the  computer  progrom  could  handle  a  total  of  40  input  and  state 
variables  the  problem  v.as  not  the  number  of  variables.  The  problems  centered  pround 

i 

the  behavior  of  the  variables.  ; 
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If  was  learned  that  a  mar-organized  sysfem  as  simulated  produces  variables  some 
of  which  cause  redundancies  in  the  equations.  This  of  course  causes  singularities 
in  the  matrix.  A  particular  variable  of  the  stimulus  data,  though  time-varying  over 
the  long  term,  may  have  a  constant  value  for  the  simulated  time  period;  and  when  the 
forward  and  backward  differences  are  taken,  the  value  of  the  difference  is  zero.  This 
same  problem  can  occur  with  a  parameter  whose  first  derivative  is  a  constant.  The 
forward  and  backward  differences  will  be  constant  but  in  the  multiple  regression  analysis, 
a  zero  variance  is  computed.  A  possible  solution  to  the  problems  of  zero  values  for 
the  variances  is  the  use  of  a  "dithering  signal, "  much  the  same  tbs  the  dithering  signal 
used  In  control  systems .  The  redundancies  in  the  equations  can  be  overcome  by  careful 
selection  of  the  system. parameters. 

After  taking  careful  note  of  the  above  conditions,  attention  is  then  directed  to 

\  .  '■  ■ 

the  types  and  levels  of  stimulus  data.  The  prediction  capabilities  of  the  DONA  model 

*  '  i 

can  be  tested  through' its  ability  to  "track"  the  real  syslcm  or,  as  in  the  case  of  the 
laboratory  concept,  t^  "track"  the  SIMCO  simulation.  The  inputs  to  SIMCO  and  DONA 

may  be  any  one  or  a  .combination  of  signals  composed  of  randpm  noise,  sinusoids,  implies 

/ 

or  step  funictions.  A  particular  characteristic  to  observe  Is  the  frequency  response  of  / 
the  DONA  model,  The  Interactions  designed  into  the  SIMCO  simulation  were  part  of 

l 

this  test  to  determine  the  "worth"  of  the  DONA  model  and  the  method  to  produce  the 
model. 

Problems  can  also  arise  in  the  comparison  phase  of  the  laboratory  concept.  The 
criterion  for  agreement  in  the  outputs  is  based  on  the  quadratic  form  of  the  covoriance 
matrix.  A  recent  book  by  Jonkin:  c::d  Wnlts^  describes  this  procedure  when  cn  onalysis 
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i.  required  of  o  multiuoriote  rytem.  The  equotlon  of  the  quodrotic  form  of  the  eo,orio„c. 
matrix  is  as  follows: 

Probability  {[X-af  V“JCX-m]}  =  Probability  {%2m) 

whore  V=£covT{yO,t),y(M)n 

m  *  degree*  of  freedom  =  order  of  V  matrix 

Thl.  equation  i.  u.«d  In  Iwo  different  Matltflcol  le.tr.  Firet,  the  equation  |,  u„d  ,0 
the  prediction  capability  of  the  model  «  o  function  of  time.  It  con  be  determined  when 
In  the  time  domoin  the  model  ceose.  to  predict  with  the  spec  (fled  confidence.  The  ! 
equotion  then  may  b,  u,«d  to  t.,t  the  ri9niflconc.  of  eoch  dlmemlon  (vorlobl.)  In  the 
model.  Initeod  of  meorurlhp  the  uo, lability  a,  o  hmctlon  of  time  the  vorlobllity  I. 
mecmred  or  e  function  of  u  udrloble  In  the  model  with  the  time  fined.  Agcln  It  con 
be  observed  when  the  model  eeoie,  to  predict  with  the  tpeelfled  confidence.  U.ln« 
this  method  for  comparison'  of  the  output,  quonlltdtlve  Informotlon  I.  .d  which 

qlde.  the  performance  chajacteri,,lc.  of  the  DONA  model  dnd  the  method  utod  to  pm- 
duce  the  DONA  model,  j 
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Distributions  on  Bayesian  Confidence  Intervals 
For  Attribute  Data 
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Aberdeen  Research  and  Development  Center 
Aberdeen  Proving  Ground,  Maryland 

ABSTRACT 

One  method  of  obtaining  confidence  intervals  on  the  reliability  of,  a 

system  or  component  whose  sample  outcomes  are  either  a  success  or  failure 

« 

is  founded  on  Bayes  theorem.  In  the  Bayesian  formulation  of  the  problem 
one  must  a8sumej  a  prior  distribution  on  the  random  variable  of  interest, 
namely,  the  reliability.  The  purpose  of  this  invest lgation*was  to  determine 
the  effect  of  some  prior  distributions  on  Bayesian  confidence  intervals  in 
which  it  was  assumed  that  the  prior  distribution  may  be  represented  by  the 
beta  distribution.  It  was  my  intent  to  restrict  attention  to  alternative 
priors  one  might  use  when  no  previous  data  or  experience  exists  on  a 
system. 

INTRODUCTION 

Of  primary  importance  to  Bayesian  statistics  is  the  attachment  of 
probabilities  to  various  possible  hypotheses,  or  in  this  case  probable 

I  <, 

reliabilities.  The  mechanism  that  performs  this  role  is  the  prior  distri¬ 
bution.  The  plausible  values  which  the  random  variable  might  take  on  do 
not  necessarily  imply  that  one  believes  the  probability  exactly,  they  are 
only  a  measure  or  rough  indication  of  what  one  tends  to  believe  are  the 

i 

most  likely  values. 

In  Bayesian  analysis  the  prior  distribution  is  combined  with  the  test 
data  to  yield  a  modified  distribution  of  reliability,  namely,  the  posterior 
distribution.  That  is,  the  information  that  we  have  on  a  component  by  way 
of  the  prior  distribution  is  updated  with  the  latest  test  results.  If  we  let 

This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 
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W  -  Reliability,  0  <  W<  1 


n  ■  Number  of  tests 
r  ■  Number  of  successes 
Chen  by  BayeB  theorem 


f (W/n,r) 


-fgf. 

•'ll 


(Oj  r/W) f  (W) _ 

r/W)f(W)dW 


where 


f <W>  ■  Prior  distribution  of  W 

i 

g(n, r/W)  -  Probability  of  observing  r  successes  In  n  trials,  given  W. 
ffW^n.r)  •  Posterior  distribution  of  W. 

In  practice  it  is  quite  common  to  assume  a  beta  for  the  prior  distri¬ 
bution.  There  are  several  reasons  for  suggesting  the  beta,  among  them  are 
(1)  its  random  variable  has  range  (0,  1],  (2)  the  beta  can  fit  almost 
any  unlmodal  or  nonmodal  distribution  for  a  r.v.  over  the  unit  interval, 
and  (3)  its  ease  in  computations.  The  beta  density  is  given  by 

f(W)  „  _i5±fctlLLwa(i-w)b,  0<Wdl 

a!  b! 

■  0  elsewhere 

where  a,b  ■'-1,  It  may  be  shown  that  by  altering  a  and  b  the  ahape  of  the 
distribution  Is  changed.  See  Figure  1.  These  distributions  indicate  the 
plausible  values  which  the  random  variable  might  take  on. 

Assuming  f(W)  to  be  beta  distributed  and  g(n,r/W)  to  be  a  binomial 

i 

distribution,  the  posterior  distribution  is  given  by 


f(W/n,r) 


(n, r/W) f (W) 
gCn,  r/W) f (W) dW 


M  r  n-r  (a+b+1)  I  „a(,  y>b 
.  ItfW  (1-W)  albl  W  U  W) 

f1  (3wIu-rt"-tisrt«U^(1.w)bJH 
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> 


(a+b+n+1)  1  ua+f  t\  ,,vb+n-r 
(a+r) ! (b+n-r) !  ’ 


where 

Fj_a»  (1-a)  percentile  of  the  F-  distribution  with  2(b+n-r+l)  and 

2(a+r+l)  degrees  of  freedom. 
a,b  *  parameters'  from  the  prior 
n  *  sample  size 
r  ■  number  of  successes 

The  use  of  the  F-  distribution  may  be  seen  from  the  following.  From 
the  change  of  variable  theorem  of  Integral  calculus  we  may  write 

g(F)  -  h(W  -  U(F) ) 
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and  on  substitution 


tv\  ■'  (a+b+n+2) _ /  a+r+1  \ 


a+r+1 


\  M  ■  L.  •  4.  /  -  \+*  *  ** 


11+ 


Fa+r _ 

L»i.1  IL  J - 1  «1 


7 


b+n-r+1 

\ 

which  Is  the  F-  distribution  with  2(a+r+l)  and  2(b+h-r+l)  degrees  of  freedom. 
Now  recall 


V 


.1 


J  f(W)  dw  •/  f  (U) dW  -  l-a 
V,  UF 


1  +  RF 


where 


awl 

b+n-r+1 


a  k 

Noting  that  Ffc(a)  ■  l  /Ffl(l-a) ,  the  lower  limit  is  given  by 


IT 


i  +  b+n-r+1 
1  +  a+r+1  Fl-a 


One  of  the  reasons  for  using  the  F-  distribution  is  that  tables  of  this 
distribution  are  frequently  more  readily  available  than  those  of  the  incomplete 
beta.  Alsu,  they  are  convenient  for  those  values  of  a  most  often  used,,  e.g., 
0.10,  0,05,  0.01. 

For  sufficiently  large  sample  sizes  the  normal  approximation  may  be 
employed,  the  approximation  being  best  in  the  vicinity  of  W  equal  to  one-half. 
For  the  posterior  distribution  the  expected  value  and  variance  of  W  are: 


E(W) 


a+r+1 

b+n-r+1 


and 

a  2  .  (a+r+1)  (b+n-r+1) 

(a+b+n+2) "(a+b+n+3) 

Thus,  the  100(l«-a)%  lower  limit  is  given  by  E(W)  -  z  0 
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where  z.  is  such  that 
L-a 


X _ 

v'TrT 


J 


z 


1  «n 


e 


■} 

-t~l  2 


—  w 


DISCUSSTION 

As  indicated  in  the  previous  section,  the  lower  confidence  limit  (LCL) 
was  computed  for  illustrative  purposes.  A  confidence  level  of  95%  was  chosen 
and  the  following  prior  distributions  were  selected: 


a 

-1  -1/2  0  1  6 

b 

-1  -1/2  0  0  1 

References  [1]  and  '[4]  give  discussion  on  the  reasons  for  assuming  prior 
distributions  when  a  and  b  are  both  set  equal  to  -1,  -1/2,  and  0.  By  letting 
a-6  and  b-1  we  give  more  credence  to  moderately  high  reliabilities,  which 
would  appear  in  be  a  likely  area  of  Interest.  For  large  a  and/or  b  one  getB 
into  the  problem  of  the  prior  distribution  far  outweighing  the  most  recent 
evidence  or  sample. 

In  Figure  2  the  lower  95%  limit  is  plotted  against  the  number  of  fail¬ 
ures  observed  in  a  sample  of  size  twenty.  In  general,  the  results  are  quite 

I 

similar  for  the  other  sample  sizes  investigated  (n»10(5)<25).  It  may  be  noted 
that  the  number  of  failures  begins  at  one.  This  was  done  since  when  using 
the  (M,  -1)  prior  one  must  assume  the  occurrence  of  at  least  one  success  and 

I 

at  least  one  failure  in  the  sample.  j 

An  examination  of  the  lower  limits  plotted  on  Figure  2  yields  the  follow¬ 
ing  general  results: 

1.  The  (-1,  -1)  prior  results  in  shorter  confidence  intervals  than 
a (-1/2,  -1/2)  prior  which  in  turn  gigas  a  shorter  confidence  interval  than 
the  (0,0)  prior.  This  holds  true  for  moderate  to  high  reliabilities.  The 
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0 


order  Is  reversed  for  low  reliabilities. 

2.  The  differences  in  confidence  interval  length  become  smaller  as 
n,  the  sample  size,  increases. 

3.  All  thre^  priors  [(-1,-1),  (-1/2,  -1/2),  (0,0)]  result  in  shorter 
confidence  intervals  than  those  obtained  with  classical  methods. 

4.  The  (6,1)  prior  does  not  result  in  a  uniformly  shorter  confidence  i 
interval  for  moderate  to  high  reliabilities. 

Now,  let  r/n  (d  success/sample)  be  an  indicator  of  rallablllty  and 

rank  the  priors  according  to  (heir  interval  length.  Thus  the  shortest 

1 

confidence  Interval  indicates  the  most  optimistic  result  and  the  longest 
the  most  pessiminstic  result  for  sample  estimates  of  reliability.  The  follow¬ 
ing  table  presents  a  summary  of  these  rankings. 

Summary  of  Length  of  Confidence!  Intervals  for 
(-1,  -1),  (-1/2,  -1/2),  (0,  0),  (1,  0)  Priors 


r/n 

0.85 

0.75-0.85 

0.65-0.75 


Shortest 

(-1,  -1). 

(-1,  -1) 

(1,  0) 


(-1/2;  -1/2) 
(1,  0) 
(-1,  -1) 


(1,0)* 

a/2;  -  1/2) 

,(-1/2,  -1/2) 


Longest 

(0,6) 

|(0,  0) 

,  (0,  0) 


Thus,  for  r/n  greater  than  0.75  the  (-1,  -1)  prior  gives  the  shortest 
Interval  while  the  uniform  prior  (o,  0)  gives  the  longest  lntsrvel. 


/" 


/ 


* 
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SOME  TECHNIQUES  FOR  CONSTRUCTINr. 
MUTUALLY  ORTHOGONAL  LATIN  SQUARES* 


VI.  T.  Fedetei  ,  A.  Hedayat ,  E.  T.  Parker 
B.  L.  Raktoe,  Esther  Seiden,  and  R.  J.  Turyn 
Mathematics  Research  Center,  U.  S.  Army 
Madison,  Wisconsin 


AlgfliRffir<i.  Various  methods  of  constructlnit  a  set  of  mutually-- 
orthogonal  lstin  squares  are  presented  and  the  theoretical  aspects 
of  various  methods  are  discussed.  Illustrative  examples  of  con¬ 
structing  latin  squares  and  sets  of  mutually  orthogonal  latin  squares 
are  given.  The  methods  of  constructing  latin  squares  and  sets  of 
orthogonal  latin  squares  are  complete  and  partial  confounding,  frac¬ 
tional  replication,  analysis  of  variance,  group,  projecting  diagonals, 
orthomorphism,  pairwise  balanced  design,  oval,  code,  product  composition, 
and  sum  composition.  The  methods  of  construction  designated  as  partial 
confounding,  fractional  replication,  analysis  of  variance,  and  sum  com¬ 
position  appssr  not  to  have  bssn  discussed  previously  in  tha  literature. 
The  methods  of  complete  confounding  and  of  projecting  diagonals  have 
been  given  only  a  passing  rafaranca  with  no  indication  aa  to  the  actual 
construction  procedure.  Tha  sum  composition  method  has  interesting 
constquencss  in  combinatorial  theory  aa  wall  as  in  the  construction  of 
orthogonal  latin  squares.  Lastly,  equivalences  of  fourteen  combinatorial 
eye terns  to  orthogonality  In  lstin  squares  has  been  investigated  and 
described. 


*Thle  article  has  appeared  as  Technical  Summery  Report  No.  1030  of  The 
Mathematic*  Research  Center,  U.  S.  Army,  The  University  of  Wisconsin, 
Madison,  sponsored  under  Contract  No.:  DA-31-ARO-D-462. 

The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  authors'  manuscript. 
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St, ML  TECHNIQUES  WjH  \  iN-STRUCTING  MUTUALLY 
ORTHOGONAL  LATIN  SQUARES 

W.  T.  f'ederer1,  A.  Ua  inya i* ,  L.  T.  Parker3 
B.  L.  Raktoe4,  Esther  .v'luen"’.  anc  R.  J.  Turyn  ’ 


I.  Introduction  and  Some  Terminology 

the  purpose  of  this  paper  is  to  present  a  set  of  methods,  for  constructing 
mutually  orthogonal  latin  squares  and  tc  exhibit  some  squares  produced  by  each 
of  the  methods.  The  set  of  methods  pe  s-.nteu  herein  v/as  discussed  in  a  series 
of  informal  seminars  held  during  the  v.'-.-rrs  July  14-lh  am  di-^5.  l.o<9.  ay  the 
authors  at  Cornell  University.  Th*-1  m<  tivution  for  thrsi.*  discussions  was  derived 
from  results  obtained  by  Hednyat  1 1'‘>  *)  r.ci  tr„m  wo  ij-tin.isr  of  the  authors. 

New  procedures  for  constructing  i  set  .  mute  illy  ort.it  yoiul  latm  squares  and 
new  views  of  present  methods  of  ;w.  »trj  ■‘•n  acytred  in  oraer  to  advance  the 

theory  of  mutual  orthogonality  in  latin  squares. 


Professor  of  Biolugical  Statistics,  ■  ^rv  ll  University  an  i  Visiting  Professor, 
Mathematics  Research  Center.  U.S.  Ar  .y  m  i  Ui.i  :sity  of  Wisconsin  (on 
sabbatical  leav>  19b9-70). 

^  Assistant  Professor,  Cornell  University. 

*  Professor  Mathematics,  University  <  !  Hhiiui.s,  at..;  Visiting  Professor,  Cornell 
University  i July .  19t»9). 

4 

Associate  Professor,  University  of  Gc-lnn  one  Visiting  Associate  Proiessor, 
Cornell  University  (January  to  August.  1'»'>'.m. 

5  Professor,  Michigan  State  Universit'.  .  ar  i  Visiting  Profi  ss,-r.  Corn*!!  Uni¬ 
versity  (June,  July,  August,  l9o9). 

b  Mathematician,  Raytheon  Coronation,  and  Visiting  Prole.-sor .  Cornell  Uni¬ 
versity  (July.  19n9). 
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As  may  be  noted  from  tne  taole  of  contents,  the  different  sections  were 
written  by  different  authors.  An  attempt  was  made  to  have  a  consistent  notation 
and  a  uniform  style.  Although  much  more  work  is  required  to  finalize  the  method 
in  several  of  the  sections  enough  is  known  about  the  method  to  use  ft  to  con¬ 
struct  a  latin  square  of  any  order  or  to  construct  a  set  of  two  or  more  mutually 
orthogonal  latin  squares.  Also,  a  number  of  equivalences  may  be  noted  for  some 
o-i  th--  methods. 

The  theory  of  mutual  orthogonality  in  latin  squares  has  application  in  the 
construction  of  many  classes  of  rxj  •  rlnu  nt  designs  and  in  many  combinatorial 
systems.  The  latter  subject  is  discussed  in  section  XV  where  the  equivalences 
of  various  combinatorial  systems  an;  presented,  With  regard  to  the  former  sub¬ 
ject.  there  is  an  ever  present  need  (nr  rvv,  experiment  designs  for  new  exporl- 

i  '  . 

mental  situations  in  order  for  the  "X{.  i imontc  r  not  to, have  to  conduct  his  ex¬ 
periment  to  fit  known  experiment  designs. 

ioiii-  ut  the  notation  and  terminology  that  will  bo  utilized  is  presented 

below. 

Definition  1. 1 .  A  latin  square  ot  i-r  :...r  n  on  o  set  ^  with  n  distinct  elements 
is  an  n  x  n  tna*trix  each  of  whose-  rows  and  columns  is  a  permutation  of  the  set 
V 


is  a  latin  square  of  order  1  on 


{l.d.3}  . 
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Definition  1.  i  .  Two  latin  squares  I  .  -  ia  >  and  L  =  lb  i  of  order  n  arc 

]  ,,  ili 

said  to  be  orthogonal  if  the  ordered  pairs  ia  ,  b  i  fi.j  =  l,Z . n)  are 

U  i  J 

all  distinct.  Note  that  Lj  and  1.^  nee,j  not  -  defined  on  tho  same  set. 
Example; 


J_ 

j 

A 

A 

fi 

c 

i 

3 

T 

and 

O' 

A 

B 

r 

1 

i 

XO 

k 

Definition  1.  3.  Th*'  members  o!  s<'t  of  t  iatin  squares  i,.  9  L . ,  •••  9  2.  cf 

•  1  it  t 

or  ier  r.  are  sai  ;  t^  bn  mutually  ipi  ir.vi.o  .  rt!.w.gonJl  if  i.  is  oithogonal  to 
1.  ,  i  •  j,  i,i  l,i . t  .  H  're, if.-  r  :>y  > rjin.ti  s-  t  we  fi  '  On  a  s-”*.  con¬ 

sisting  of  t  putmlly  orthogonal  It  tin  i-  r-  s  <.  :  >rd  ■  t  r.  . 

Example; 


r r 

l 

l 

4 

1 

"1 

i 

Z 

4 

i 

» 

1 

i 

t 

4 

3 

1 

z 

..  . 

•4 

TT 

i 

L- 

! 

Z 

i 

4 

lJ- 

TP 

4 

i 

r 

> 

i 

4 

2' 

T 

4 

3 

i 

4 

T 

Z 

T 

3 

~r_ 

T 

Latin  squares  and  orthogonal  latin  r.quari.s  nave  at  least  18V  years  of  history. 
Hedayat  ( 19  by- section  IX]  has  pres  on  to  i  a  reasonably  good  picture  of  this 
history  which  will  not  be  repeated  here.  It  m  planned  to  prepare  «  historical 
account  of  developments  related  to  orthogonality  in  latin  squares  and  to  publish 
this  material  together  with  a  bibliograph/  nl.u  wh'i" 
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ii.  i.  Compile  Confounding 

The  r/  row-column  intersections  may  bo  related  to  the  treatment  combin¬ 
ations  in  a  n  factorial  treatment  design.  To  illustrate  let  us  consider  the  4* 
factorial  and  the  latin  square  of  order  q-."  The  levels  of  the  main  effects  ,  A 
and  B.  in  the  factorial  will  b<-  used  to  designate  the  rows  and  the  columns  of 
t r, •  latin  square  .,f  order  4  as  f,  1!  ws; 


lot  in  squares  lI  oruet  4 


Thu  ;,  four  cumoir.ations  out  of  the  l><  wuch  have  i  =  0  in  the  subscript  ijf 
1.0'.,  00,  <J  1  ,  i)2,  and  03.  art  designated  as  iAiq  and  are  pul  in  row  1,  Con¬ 
tinuing  this  procedure  the  remuind<  r  of  the  ij  combinations  are  allocated  to  the 
remaining  rows  and  to  the  columns  as  sh  .wn  above. 

Now,  three  other  effects  with  4  levels  each  can  be  set  up  from  the  pro- 

2  Ul  U2 

jective  geometry  PGil.2  i;  the  effects  are  tAR  )  ,  (AB  )  .  and 

u.  -u.u  u  * u ,u  ’ 

0  J  1  1  J  i  2  j 

(Ah  i  .  The  levels  of  these  effects  and  the  corresponding  latin  square 

u  *u  ,u 
1  3  I 

produced  by  letting  all  combinations  of  the  level  of  an  effect  be  a  symbol  in  the 
luttn  square  are  (see  page  3  37  of  Kempthorne  [  Lli62]  ,  e.g.  i; 
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0  00  +  i  1  +  22  +  3  3  *  1 
1  01  +  10  +  23-*  32  *11 


lAD  ‘l  -  t 

Vulu,  I 


2  02  4  1  3  +  20  +  31  *  111 

3  03  +  12  +  21  +  30  *  IV 


oo=:  ; 

01  =  11 

02  -  III 

0  3-  IV 

10=:  JI 

1  I  -  I 

12  =  IV 

1  3=  III 

20=  111 

2  1  =  IV 

22  =  1 

2  i=  11 

30=  IV 

31=111 

32=  11 

33=1 

u2 

(AS  l) 


VU2V 


‘0  00  +  13  +  21  +  32  -  a 

1  03  +  10  +  22  +  31  -  (3 

2  01  +  12  +  20  +  33  -  v 

3  02  +  11  +  23  +  30  *  6 


a 

V 

6 

p 

6 

Y 

a 

-  - . 

v 

a 

(3 

6 

6 

P 

a 

Y 

U3 

(AB 


Ui+U  3Uj 


C  0  00  +  12  +  23  +  31  •  W 

|  1  02  +  10  +  21  4  3  3*  -  X 

2  03  +  11  +  20  +  32  *  Y 

3  01  +  13  +  22  *  30  *2 


W 

Z 

X 

Y 

X 

Y 

w 

Z 

Y 

X 

z 

W 

2 

w 

Y 

X 

In  the  above  the  complete  confounding  scheme  of  sources  of  variation  in 
the  0(4,3)  set  and  the  effects  in  the  factorial  may  be  illustrated  in  the  following 
analysis  of  variance  table  wherein  the  sums  of  squares  in  the  lines  of  the  analysis 
of  variance  are  orthogonal  to  each  other: 


Source  of  variation 
Correction  for  mean 

Rows  =  A -effect  * 

Columns  =  B  effect 


Ji. 


Roman  numbers  =  (AB  )  effect 
u2 

Greek  letters  ■  (AB  )  effect 
u3 

Latin  letters  *  (AB  1  effect 


Degrees  of  freedom 
l 

3 

3 

3 

3 

3 


/ 


/ 


Total 


I  6 


679 


0 


Instead  of  relating  the  mutually  orthogonal  latin  squares  of  order  4  to  a 
-l  factorial  we  may  relate  them  to  a  l  factorial  in  the  following  manner.  Let 
the  16  row-column  intersections  bo  numbered  as  follows} 


column 


row 

1 

“2 

J . •' 

. . — H 

1 

0000 

0001 

0010 

0011 

Z 

0100 

0101 

0110 

0111 

i 

moo 

1001 

1010 

1011 

4 

-1100 

i  101 

1  1  10 

nil 

Let  the  factors  be  a,  b.  c,  and  1  with  two  levels  (0  and  1)  each.  The  rot/s 

\ 

correspond  to  factorial  effects.  A.  B.  and  AB  and  the  columns  correspond  to  fac¬ 


torial  eftects  C,  D,  and  CD  .  ilhis  form  of  constructing  latin  squares  has  been 
usod  by  Usher  and  Yates  1 1*967)  for  latin  squares  of  order  H.  )  Then,  let  the 
symbols  in  the  3  latin  squares  be  represented  by  the  following  scheme: 


i  actoridl  generators 

(AC l.  .  (BD>„,  (ABCDi, 
0  0  U 

<ACi(),  1 B D i j  ,  (ABCD*! 
(A Ci  iBD!0,  (ABCDi  ( 
(AClj  ,  iBDij  ,  (ABCDi0 

(ADi().  (ABCi0,  (BCD)0 
(ADi().  (ABC) ^ ,  (BCDij 
(ADl j  ,  (ABC>0,  (BCD) j 
iADi(  ,  (ABC ) | ,  (BCLiIq 


O  unbinations 

000U  >  0  10  1  4  it)  1 0  -  1111  = 

oooi  •  oioo  •  ion  *  mo  = 

ooio  +  0111  -  1000  -(  noi  = 

0011  4  0110  -  1001  ->  1100  = 

0000  *  0110  *  1011  t  1101  = 

ooio  •  oioo  *  mo i  i  1 1 1 1  = 

oooi  ■*  om  a  ioio  ♦  noo  = 

0  101  +0011*  moo  +  1110  = 


latin  squares 


i 

1 

11 

in 

IV 

ii 

II 

I 

IV 

ill' 

m 

III 

IV 

i 

a 

IV 

IV 

HI 

ii 

i 

w 

W 

Z 

X 

Y 

X 

X 

Y 

w 

Z 

2 

Y 

X 

z 

W 

Y 

1 

W 

Y 

X 
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(ACD)fl,  (BC)0,  (ABD)0 
(ACD)0,  (BCI,,  (ABD)t 
(ACD),,  (BC»0,  (ABP), 
(ACD)j ,  (BC), ,  (ABD)0 


0000  40111  *  11104  1001  =  a 
1010  4  0100  4  001  1  +  1101  =  (J 
1000  +  0110  +  1  1  1  1  +  0001  =\ 
0010+  0101  4  1011+1100=  6 


.  The  correspondence  of  the  latin  squares  obtained  from  complete  confound- 
ing  considering  a  4  factorial  and  considering  a  2  factoria  l  "is  demonstrated  - 

in  the  following  analysis  of  variance  table: 


Source  of  variation 


degrees  of  freedom 


Correction  for  mean 


=  A  effect  in  4  factorial 

4 

A  effect  in  2  factorial 
. 4 


Columns 


B  effect  in  4  factorial 


C  effect  in  2  factorial 
^  . 4 


1  2 

Roman  numbers  =  AB  effect  in  4  factorial 


AC  effect  in  2  factorial 


ABCD  "  "  2 


2  2 

Greek  letters  =  AB  effect  in  4  factorial 


ACD  effect  in  2  factorial 


ABD 


1 


•  2 

Latin  letters  =  AB  '  effect  in  4  factorial  3 


s 

AD  effect 

4 

m  2  factorial 

1 

ABC 

4  " 

"  2 

1 

BCD 

i,  ^  ^  ii 

1 

Total 

16 

It  should  be  noted  here  that  the  effects  In  the 

4 

2  map  directly  into  the 

2  2 

4  projective  geometry  or  PG(1,2  i  , 

Likewise,  even  though  one  more  set  of 

generators  is  available,  viz. 

Generators 

interaction 

Roman  numbers 

-  AD,  BC 

ABCD 

Greek  letters 

--  AC,  ABD 

BCD 

Latin  letters 

--  BD,  ABC 

ACD 

the  three  orthogonal  latin  squares  produced  ore  the  same  ones.  Since  the  third 

effect  above  is  obtained  as  the  product  of  the  two  generators,  mod  2,  we  need 

consider  only  the  generators.  Multiplying  these  by  CD  (mod  2)  we  obtain  the 

generators  of  the  preceding  scheme.  Hence,  even  though  two  different  complete 

confounding  schemes  are  available  there  is  a  simple  one-to-one  mapping  of  one 

set  into  the  other  sot.  Although  nothing  interesting  turns  up  here,  it  would  be 

interesting  to  study  the-  various  complete  confounding  schemes  in  the  latin  square 

4 

of  order  t  as  related  to  the  3  factorial. 

As  a  second  illustration  of  the  use  of  complete  confounding  to  construct 
latin  squares,  let  us  consider  the  latin  square  of  order  6.  Using  the  notation 
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20 

2 
2 
2 
2 
2 
2 
2 
2 
2 
2 

Total  36 

Let  us  now  set  up  the  6  rows  and  the  6  columns  of  a  latin  square  of  order 
6  with  the  corresponding  designation  of  the  36  combinations  as  follows t 


Columns 


Rows 

IB3d‘,0 

(B3d4)1 

cbVi,, 

<b3d4)3 

<bjd\ 

(b3d4)5 

(AJC\ 

0000 

0304 

0002 

0300 

0004 

0302 

(A3c4)1 

3040 

3344 

3042 

3340 

3044 

3342 

(a3c4)2 

0020 

0324 

0022 

0320 

0024 

0322 

(A3C4)3 

3000 

3304 

3002 

3300 

3004 

3302 

(A3c\ 

0040 

0344 

0042 

0340 

0044 

0342 

(A3c4,5 

3020 

3324 

3022 

3320 

3024 

3322 

Remainder 

cV 

A3X  d4 
A3  X  C4D4 
A3  X  C4D2 
B3  X  C4 
B3  X  C4D4 
B3  X  C4D2 


A3B3X  C4 

a3b3x  d4 

A3B3  X  C4D2 
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3  3  4  4 

Now  let  the  levels  of  A  B  C  D  correspond  to  the  symbols  In  a  latin 
square  of  order  6  as  follows  t 

Levels  Combination  for  which  3q*3h+4it4i.  mod  fe.  is  constant  Symbol 


(A3B3C4D4)0' 

0000  4 

3342  4  0024 

4  3300  4  0042  4  3324 

♦ 

0 

(A3B3C4D4)1 

0304  4 

3040  4  0322 

4  3004  4  0340  4  3022 

-w 

1 

3  3  4  4 

<aVcV)2 

0002  4  3344  4  0020 

4  3302  .4  0044  4  3320 

-* 

2 

(A3b3c4d4>3 

0300  4 

3042  4  0324 

4  3000  4  0342  4  3024 

* 

3 

(A3B3C4D4)4 

0004  4 

3340  4  0022 

4  3304  4  0040  4  3322 

• 

4 

<a3b3c4d4)5 

0302  4 

3044  4  0320 

4  3002  4  0344  4  3020 

•  * 

5 

This  produces  the  following  latin  square  of  order  61 


0 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

0 

2 

3 

4 

3 

0 

1 

3 

4 

5 

0 

l 

2 

4 

5 

0 

1 

2 

3 

5 

0 

1 

2 

3 

4 

Alternatively  wo  could  have  used  levels  of 


.3_3_4_2 
A  B  C  D 


to  construct  the 


following  latin  square  of  order  fei 

Levels  Combinations  for  which  3g4  3h44l42j ,  mod  fe.is  constant  Symbol 


<a3bW)0  0000 

(A3B3C4D2).  0302 

<aYcV)2  0004 
(A3B3C4D2)3  0300 

(A3B3C4D2)4  0002 

(aVc4D2)5  0304 


4  3344  4  0022  4 
4  3040  4  0324  4 
4  3342  4  0020  4 
4  3044  4  0  322  4 
4  3340  4  0024  4 
4  3042  4  0320  4 


3300  4  0044  4  3322 
3002  4  0340  4  3024 
3304  4  0042  4  3320 
3000  4  0344  4  3022 
3302  4  0040  4  3324 
3004  4  0342  4  3020 


0 

1 

2 

3 

4 

5 


6B5 


latln  square  of  order  A 


0 

5 

4 

3 

2 

1 

1 

un 

3 

4 

3 

2 

2 

1 

0 

5 

4 

m 

3 

2 

1 

0 

5 

tm 

4 

3 

2 

1 

0 

3 

3 

4 

3 

2 

1 

0 

Thus,  tho  above  square  is  simply  a  column  permutation  of  the  previous  one.  As 
inure  are  no  other  sets  ut  5  degrees  of  freedom  leading  to  a  latln  square  of  order 
A  li.  e.  A3,  B3,  and  A3B3  exhaust  the  three  single  degrees  of  freedom  from  tho 

2  4  A  4  4  42 

2  factorial  and  C  ,  D  .  G  D  ,  and  C  D  exhaust  all  sets  of  2  degrees  of 

freedom  from  the  3*“  factorial  u  It  is  not  possible  to  obtain  a  latln  square  of 

order  n  orthogonal  to  either  of  th'‘  preceding  ones  using  complete  confounding 

schemes, 

for  a  latin  square  of  order  10  we  may  use  levels  of  A^C^D*1,  A5BiC<’D‘’t 
A^b’c^D*,  or  A3B5C^D4  to  form  (out  different  latln  squares  of  order  10, 

II.  2,  Partial  Confounding 

In  the  last  section  use  was  made  of  complete!  confounding  of  effects  In  a 
factorial  with  the  rows,  columns,  unu  symbols  in  a  latin  square.  Instead  of 
completely  confounding  an  effect,  it  could  be  partially  confounded,  for  example , 
the  latln  square  of  order  4  could  be  considered  as  a  24  factorial  as  In  the 
preceding  section,  with  the  following  scheme  of  confounding: 
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r. 


i  =  iCL 

u 


3  =5  ( D)„ 

U 


1  l'VIB'o 

0000 

00  1  1 

0010 

*  l 

i 

(A)0,(B)l 

0101 

0  110 

0100 

i 

3 

(AJj  i  ( B >,j 

1000 

10  11 

1010 

-4 

(A),  ,  (B), 

1101 

1110 

1  100 

4  *  (Dl 


0 


If  we  set  up  the  latin  square  symbols  for  the  above  as 


N  ft  ti 
e  >,  a. 


the  symbols  correspond  to  the  following  combinations: 

Hi  nooo  4  OilO  ■»  1001  4  1  !  oo  =  lABCD>(  ■*  other  effects 

0 1  0011  I  010  1  4  1010  -  INI  -  lAtiCD)  * 

1000  -  I  I  10  ■*  0010  *  0  1  I  I  a  (ABCDi  + 

hi  0001  *  0100  4  101  1  *  1  10  !  -  lABCDlj  • 

It  is  known  that  this  latin  square  has  no  orthogonal  mate  (Hedayat  |  1U60J 
This  means  that  no  .orthogonal  partition  of  the  remaining  sum  of  squares  can  be 
madi>  which  forms  a  latin  square*. 

o  |  ti  ^  «S 
^  *»  •)  "\ 


If  on  the  other  hand,  the  latin  square  used  is 


,  the  uomtjiiidtions 


corresponding  to  the  Greek  letters  are: 

0000  -  0110  ♦  1010+  1111  =  (ABCD)0  +  other  effects 

001  1  4  0101  4  100  1  +  1100  -  fABCD)0  +  . 

•y:  0010  +  0111  +  1000  +  1  1  10  =  {ABCDlj  +  (AC)j  +  other  effects 

bi  000  1  +  0100  +  1011  +  1101  =  (ABCD)  +  other  effects 

1ms  square  has  two  mutually  orthogonal  mates  and  hence  there  must  be  partitions 
.,f  mi'  sums  of  squares  nto  orthogonal  components  which  correspond  to  the  symbols 

i .  i  1>J  1 1  .’i  U  d  T O  ♦ 

instead  of  inserting  symbols  in  the  latin  square  of  order  4,  denote  the 

symbols  in  the  latin  square  by  the  following  partial  confounding  scheme: 

ii  add  the  two  1/8  replicates  generated  by  ((A)^,  iD)^,  and 

1 1 A  l ,  .  id  ,  iABDi^i  to  obtain  the  4  combinations  (0000  +  0110)  - 

i loio  +  llili  and  denote  these  4  combinations  as  symbol  a 

hi  add  th>  two  1/8  replicates  generated  by  MDij,  lAB)^  (AClgJand 

. i A h *  j (AL)y  to  obtain  conibmations  ( 0  lo  1  *  loll)  *  1 11 00  * 

onoli  and  denote  these  4  combinations  as  symbol  B  , 

iin  aad  the  two  1.  K  replicates  generated  by  (<A)j,  '  0)Q ,  (ABC)^i  and 

1 1 A  -  ,  iC)  ,  iBDui  to  obtain  combinations  (looo  +  1110)  +  (0010  + 

0  1  0 

■  1111 1  hkJ  denote  these  4  as  symbol  y  , 
iv.  id u  the  two  1/8  replicates  generated  by  i(ABiq,  (AC)j,  (D)^»  and 
iiABy  >  ^ * q »  (BDy  to  obtain  the  combinations  <1101  +  0011)  + 

. r: loo  *  looli  and  denote  these  4  as  symbol  5  . 
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This  procedure  results  in  the  following  latin  square  of  order  4: 


a 

6 

a 

6 

\ 

y 

a 

6 

6 

V 

p 

at 

Obviously,  one  could  take  any  pair  of  1/8  replicates  such  that  the  4  combina¬ 
tions  are  in  different  rows  and  in  different  columns  to  form  the  combinations  for 


a  given  symbol. 

The  above  type  of  partial  confounding  results  in  the  class  of  latin  squares 
denoted  as  half-plaid  latin  squares.  If  partial  confounding  were  utilized  in  rows 
as  well  as  in  columns  the  resulting  square  would  be  denoted  as  a  plaid  latin 
square  (so-called  because  of  its  resemblencc  to  plaid  cloth  If  the  effects  con¬ 
founded  were  of  different  colors).  The  three  types  of  squares  are  illustrated 
below  for  a  latin  square  of  order  (>t 


Jumuletv  confounding  ot  effects 
Columns 


Rows 

i  = 

(A)0,(C)0 

2  = 

(AIq.ICIj 

3  = 

(A)g,(C>2 

4  = 

(A)lf(C)0 

5  = 

(A)j,  (C)j 

6  = 

(A>2 ,  (c>2 

1  =  (B)0,(O)0 

0000 

ooio 

0020 

1000 

1010 

1020 

mmm 

0001 

0011 

0021 

1001 

101  1 

1021 

3=  (B)q,(D)2 

0002 

0012 

0022 

1002 

1012 

1022 

4  =  (B},,(D)0 

0100 

0110 

0120 

1100 

1110 

1  120 

5  =  (B)j ,  (D)j 

0101 

0111 

0121 

1101 

1111 

1121 

asm 

0102 

0112 

0122 

1102 

1112 

1122 
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Partial  confounding  of  effects  with  columns 


Oulu  in  11 


Rows 

1  --  lC>0 

2  =  (C) 

3  =  (C)2 

4  *  (CD)0 

5  =  (CD)  j 

6  =  (CD)2 

1  ■  "VD'o 

0000 

oo  in 

0020 

.1000 

1010 

1020 

^  *  fB)0,{D>( 

0001 

oo :  l 

0021 

0002 

0012 

0022 

1  100 

1  1  10 

1  120 

0100 

0110 

0120 

32120131 

1  101 

i  1 1 1 

1121 

0121 

0111 

WEfim 

1  102 

1  1  12 

1  122 

01  12 

0122 

0 102 

Partial  confounding  in  both  rows  and  columns 
Columns 


L 


j  Rows 

'  *  '4V> 

-T=l£7j  ' 

-rrr^ 

TTTcnr^ 

T*= . (£fE5” 

1 

1 

00 

lo 

do 

00 

10 

• 

20 

2  ■  (D)| 

01 

1 1 

21 

21 

01 

1 1 

3  =  ,D)2 

02 

12 

22 

12 

22 

02 

•*  *=  (CD^lg 

■  00 

i  i 

22 

00 

22 

•i  l 

6  •«  (CD^ l j 

02 

10 

21 

21 

10 

02 

01 

12 

20 

1  2 

01 

20 

In  th'->  last  table  above  only  th < •  subscripts  for  combinations  of  factors  c 
and  d  have  boon  inserted.  There  is  some  difficulty  in  Inserting  subscripts  for 
factors  a  and  b  such  that  these  effects  are  orthogonal  to  both  rows  and  columns. 
In  any  event,  this  problem  require:;  further  study  to  determine  if  half-plaid  lattn 
squares  ar.d  plaid  latin  squares  lead  to  iatin  squares  not  of  the  same  type  as  given 
by  complete  confounding.  If  tne  three  types  ot  latin  squares  of  order  6  can  be 
produced  by  partial  and  complete  confounding ,  this  would  be  an  interesting  result. 
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III.  Fractional  Replication  Construction  of  O(n.t)  Sets 

Any  latin  square  may  be  considered  as  an  n  *  fraction  of  an  n3  factorial 

where  the  rows  represent  levels  of  one  factor,  the  columns  represent  the  levels 
,  t't 

of  the  second  factor,  and  the  symbols  in  the  latin  square  represent  the  levels  of 
the  third  factor.  As  an  illustration,  consider  the  latin  square  of  order  3  where 

« 

the  9  combinations  represent  the  1/3  fraction  of  a  3  factorial  as  follows: 


Columns 


Rows 

0 

1 

2 

0 

000 

012 

021 

1 

102 

1  1  1 

120  • 

2 

201 

2  10 

22  2 

The  above  Is  the  1/3  fraction  of  a  3 3  corresponding  to  (ABC),  , 

h+i+j  =  0,mod3 

Since  this  is  a  regular  fraction  we  may  write  out  the,  aliasing  structure  In  this 
fraction  as  follows: 

M  •  ABC 

A  i  AB2C2  BC 

2 

B  +  AB  C  4  AC 

C  +  ABC“  -f  A!3 

l  l  l 

AB  +  AC  '  3C 

where  the  effects  connected  with  a  plus  sign  are  completely  confounded  with  each 
other.  In  the  above  latin  square  the  symbols  0,1, 2  correspond  to  the  levels  of 

the  third  factor,  c.  Now  if  we  set  up  a  second  latin  square  in  which  the  symbols, 

2 

say  correspond  to  the  levels  of  AB  ,  the  resulting  square  will  be 

2 

orthogonal  to  the  first  one.  The  square  corresponding  to  levels  of  (AB  ).  +  ^  3 
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Jr* 


000  ♦  1  1  1  4  Zll  -  a 

n>  i  i  21 0  !  '02  -  |’ 

20-1  4  012  +  120  -  > 


a 

i* 

■1 

K 

u 

Y 

\ 

p 

Q 

The  class  of  fractional  replicates  constituted  as  an  nj  fraction  of  an  n 
>  ^ 
factorial  oecomes  an  important  one  to  study  as  it  relates  to  construction  of  mutu- 

’  -  3  9 

ally,  orthogonalafcrtin  squares..  In.  particular-,  ally  2  ..  fractions  of  a  2-  and  all 

'■>  “  fractions  of  a  3  with  all  possible  aliasing  structures  could  produce  several 
sets  of  mutually  orthogonal  latin  squares.  This  could  have  interesting  conse¬ 
quences  m  finite  geometry. 

The  structure  of  tne  left-hand  sot  "f  parameters  in  an  aliasing  structure 
will  have  <»  pattern;  for  example,  for  n  =  4,  5,  and  7,  the  patterns  are: 


n  =  4 

n  =  6 

n  =  7 

M  ‘  AB  id 

M  ^  ABC 

M  4  ABC 

A  ' 

A 

A 

13 

13 

B 

C 

0 

C.  -■ 

AB2 

ab“ 

AB2-  , 

AB? 

AB  3 

AB3 

4 

,  4 

AB 

AB 

5 

AB 

6 

AB 

K  :>■  that  although  ABC  was  completely  confounded  with  the  mean,  any  one  of 

U  V 

th*-  other  three- factor  interaction  components  AB  C  ,  u , v  =  l,2(..,,n-l  could 
r/.'  been  utilized  equally  well.  Also,  note  that  the  levels  of  C  corresponding 
t.  •  tyr:  nols  produce  a  laun  square  .  and  that  the  levels  of  effects  below  the 
mot,.:  A  produce  -a  sot  of  n  -  1  mutually  orthogonal  latin  squares. 

In  general  we  want  to  look  at  all  possible  n  3  fractions  of  an  n3  factorial, 

f  r,  3\ 

i.  •  .  ,  th. ■  3ubset  of  I  combinations  for  which  the  levels  of  C  are  the  symbols 
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in  a  latin  square  and  to  study  their  patterns  especially  for  n  =  7,8,  and  9.  All 
possible  fractions,  or  rather  all  forms  of  the  aliasing  structure,  could  be  classi¬ 
fied  into  all  types  of  t  mutually  orthogonal  latin  squares,  O(n.t)  for  t  =  1,2, 

. . .  ,n-l  .  Perhaps  this  is  the  manner  in  which  the  geometries  of  various  values 
of  n  can  be  exhaustively  studied.  In  fractional  factorial  notation  we  want  to 
study  all  possible  patterns  of  in  the  following  matrix  equation: 


where  the  form  of  the  first  vector  below  the  letter  C  will  be; determined  by  the 

values  in  X^X^,  the  candidates  for  entry  in-'fhe  vector  |JQ'  are  the'remaining 

■  •  2  3 

two- and  three-factor  interactions,  and  ys  is  the  particular  set  of  n  out  of  n 
combinations  for  which  the  levels  of  any  fourth  effect  in  the  first  vector  form  a  i 
latin  square.  Thus,  it  becomes  important  to  study  the  properties  of  X^a^ 
even  for  the  2™  system.  The  irregular  fractions  would  appear  to  be  the  most 
interesting  for  n  =  7,8,  and  9  since  regular  fractions  can  be  related  to  complete 
confounding  in  section  II.  1  and  to  flats  and  points  in  tho  projective  geometry. 

We  now  wish  to  illustrate  the  use  of  fractional  replication  procedures  to 
construct  latin  squares  which  are  mateless  and  which  have  orthogonal  mates.  To 
illustrate  let  us  consider  the  four  standard  latin  squares  of  order  4  which  are 
(fisher  and  Yates  [1957]): 
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& 

71 

r. 

n 

B 

1 

D 

A 

c 

r.> 

A 

B 

D 

A 

B 

C 

Square  I 


A 

R 

C 

71 

B 

D 

A 

c 

C 

A 

1) 

B 

n 

C 

m 

H 

Square  II 


"T 

m 

c 

D 

B 

D 

□ 

q 

C 

D 

B 

D 

D 

C 

□ 

B 

Square  III 


A 

m 

f^l 

”5" 

B 

a 

□ 

c 

C 

□ 

B 

B 

D 

13 

B 

a 

Square  IV 


It  is  known  (Hodayat  [  19  A?  ]  ]  that  the  dust  three  squares  are  mateless  and  that 
the  last  square  belong::  to  an  0(4,3)  set. 

N  -w  numb>T  the  -owe  a:;  !',  I,.:,  t  and  denote  these  as  levels  of  the  factor 
a;  number  the  columns  as  0,1, d, 3,  and  as  0,l,d,3  for  A,B,C,D,  respectively, 
and  denote  these  as  levels  of  the  factor  c  .  Then,  in  factorial  notation  the  above 
lb  combinations  form  a  one-fourth  fraction  of  a  4  3  factorial  treatment  design. 

The  aliasing  scheme  for  the  fractional  replicate  given  as  square  IV.  is 

M.  i  ABC 

At  RC  ■  AR^f^  *  AB  3  C 3 
B  t  AC  *,  AbJC  -t  AB*C 

I  j  ^ 

C  AB.  -*  ABC  +  ABC 

v/hnr--  the  •'•fleets  connected  with  a  plus  sign  ait;  completely  confounded  with 

•  ’ i  * ; i •  ■ : .  The  completion  of  the  p-.  lining  two  aliasing  structures  results  in 

th'-  -n,j>l. •'.'■•  a  In  sing  structures  fur  this  4  friction  of  the  4  ‘factorial!  these 
tw.  ire; 

,,C  .  •  ..  I  3  l 

/f.  -  t\  ■  1  ME 

.  i  .  d  i  d  3 

AB  •  AC  i  B  :  *  AB  C 

,)  ? 

:r  .••■•  trie  Iw-l:-.  of  All"  ur..i  'A  Ah  '  to  form  two  latin  squares,  these  two 
•.Vi* r-  -,q ,...;••  !  V  .  i id  -i , .  Oi4,  ii  sot  of  mutually  orthogonal  latin  squares. 
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/ 


Now,  let  us  return  to  the  set  of  four  standard  squares  given  above  and 
wp  nntp  rM t  only  ?icnc  in  *'\\i*Ta  IV  to  ottsin  scj’jcir^s 

I,  II,  apd  III.  These  are: 


additional  combinations 

combinations  replaced  in  IV 

Square 

I 

112,  130,  310,  332 

1  10,  1  32,  31  2,  330 

II 

II  ! 

113,  120,  210,  223 

1  10  ,  123,  21  3,  220 

- II 

ill 

213,230,326,331 

220,  231,  321,  330 

The  aliasing  structure  (v/itfcr6ut  the  coefficients)  is  given  on  the  following  page  for 
all  four  standard  latin  squares  of  order  4.  The  1/4  replicate  given  by  square  IV 
forms  a  regular  fraction/  The  remaining  three  fractional  replicates  are  such  that 
none  of  the  additional  effect's  are  unccnfounded  with  the  effects  A,  B,  or  C  of 
the  original  latin  squares  of  order  4.  Since  this  is  true  no  linear  combinations 
of  these  effects  will  be  inconfounded.  In  order  to  form  a  latin  square  which  is 
orthogonal  to' the  given  o  ie  it  is  necessary  that  there  be  a  set  of  effects  which 
is  unconfounded  with  th^ effects  in  the  given  square.  This  Is  Impossible  for  the 
three  squares  I,  II,  and  III  and  hence  the  squares  are  mateless  as  is  well-known. 

It  would  be  interesting  to  ascertain  the  aliasing  structures  for  the  six 
standard  latin  squares  of  order  5  belonging  to  the  0(5,4)  set  and  for  the  fifty 
standard  latin  squares  of  order  5  for  which  are  known  to  be  mateiess  (Hedayat 
[1969]  ).'  After  a  study  of  these  fractions,  one  should  continue  such  a  study  for 

n  -  7,  8,  and  9.  It  is  suggested  that  one  consider  a  2  fraction  instead  of  a 

3-1  9-3  3-1 

4  fraction  for  n  =  4  and  a  2  fraction  instead  of  an  8  fraction  for 

n  =  8  .  The  reason  for  this  is  that  there  is  much  more  theory  available  for  the 
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Aliasing  structure  of  effects  in  the  four  1/4  fractional  replicates 
of  a  4 3  factorial  for  four  standard  latin  squares  of  order  4 
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s  =  2  in  the  sm  series  than  for  any  other  value  of  s  ,  Also,  one  may  use  the 
generalized  defining  contrast  which  has  been  developed  by  Raktoe  and  Federet 
[1969]  to  a  considerable  advantage  in  writing  out  aliasing  structures  in  these 
cases.  Investigation  of  the  regular  and  irregular  fractional  replicates  obtainable 
for  various  values  of  n  could  lead  to  considerable  advances  in  the  theory  of 
mutually  orthogonal  latin  squares. 


IV.  ANOVA  Construction  of  Q(n,t)  Sets 

There  should  be  some  procedure  which  would  utilize  the  orthogonality  of 
single  degree  of  freedom  contrasts  in  the  analysis  of  variance  (ANOVA)  and 
which  could  be  utilized  to  construct  orthogonal  latln  squares.  For  example,  one 
could  make  use  of,  orthogonal  polynomial  coefficients  for  row  And  column  con¬ 
trasts  and  then  construct  mutually  orthogonal  latin  squares  from  these.  To  illus¬ 
trate,  consider  the  latin  square  of  order  4  used  previously  wherein  the  row-column 

4 

intersections  are  numbered  as  a  Z  factorial,  1.  e,  : 


Column 


Row 

'l - 

i 

i 

H 

1 

0010 

0011 

i 

.  m  no 

0101 

01  10 

01  1  1 

'  1001 

1010 

m 

t  10  1 

— 

1  I  10 

m 

The  relation  between  the  16  contrasts  using  orthogonal  polynomial  coefficients 
and  the  Z  factorial  Is  given  below: 
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The  individual  degree  of  freedom  contrast  matrix  for  the  above  16  comoinj* 
tions  1st 
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The  corresponding  single  degree  of  freedom  contrast  matrix  for  the  2  factorial  iss 


The  particular  contrast  matrix  utilized  is  not  unique  as  has  been 
demonstrated  above.  All  orthogonal  contrast  matrices  resulting  in  latin  squares 
could  be  considered.  For  example,  other  sets  of  contrasts  among  rows  (or  columns) 
could  be: 


1 

2 

3 

A 

1 

2 

3 

4 

Mean 

+ 

*4 

+ 

4 

Mean 

4 

4 

+ 

4 

Ri 

R2 

- 

• 

0 

0 

Rl 

- 

4 

0 

0 

0 

0 

- 

+ 

or  R2 

+ 

4 

-2 

0 

r*\ 

cc 

+ 

+ 

- 

- 

R3 

+ 

4 

+ 

-3 

The  interaction  of  row  and  column  contrasts  possibly  could  be  utilized  to  allocate 
thr>  symbols  in  the  latin  square. 

We  wish  to  illustrate  the  method  of  constructing  latin  squares  using 

i 

orthogonal  polynomial  coefficients.  We  shall  first  consider  the  construction  of 

* 

three  mutually  orthogonal  latin  squares  of  order  .4  and  then  we  shall  consider  the 
construction  of  a  single  latin  squartj  of  order  6.  In  the  preceding  table  on 
orthogonal  polynomials  for  n  =  4  denote  all  combinations  with  a  plus  sign  as 
belonging  to  (RjCjJj  and  tl?c>sp  wlth  a  minus  sign  as  belonging  to  (RLCL>0  • 

■t 

Do  likewise  for  the  RQCQ  and  effects.  Then,  the  four  latin  square 

symbols  are  obtained  as  follows: 


(Rlcl>1,  (RqCq)^  (Rccc)1  =  oooo  +  0101  +  1010  +  1111  =  a 

(P.,0,  '  »  •  ,R  c  ]  -  0001  +  0100  4  1011  +  1110  =  B 

L  Li  gg.0'  <juu 

(RLcL)0,(R0cp)o,(RcCc*r  0010  4  0,  11  4  1000+1101  sC 

(RLCL,0’  1RqCq^»  (RcCC)0  =  0011  +  01  10  +  1001  +  1  100  *  D 


This  results  in  the  following  latln  square  of  order  4 


Likewise,  if  we  use  the  following  polynomial  contrasts  we  obtain  the  two  mutually 
orthogonal  mates  of  the  above  squaret 

(RLGQ)l,  (RqCc)0,  -(RcCl10  f  0001  +  0110  +  1000  4  111  1  =  0 

(*LCQ>r  (RQcc>r*  ^c0!^  8  0010  4  0101  4  10<l1  + 1100  ”  •* 

(Rlcq)0,  (Rqcc)0,  (Rccl)1  *  0011  4  0100  4  1010  4  1  101  =  > 

( Rlcq >0 »  <RqCc>1'  (RCCl'o  =  01  1  1  +  1001  +  1,  10  +  0000  8  "'6 

and 

(RlCcJ.,  (RqCl)1,  »  0011  4  0101  t  1000'+  1110  =  I 

(RLCoV  (Rqcl)o»  ^Wo  *  0001  4  0111  +  1010  +  1100  "  11 

(RLGc)o  '  (RqCl)0»  (RCCq'i  ■  0000  4  01104  HU  4  1101  -  III 

(RLCc'o'  (RqclV  (Rccq‘o  =  0010  4  0100  4  1001  4  111  1  8  1V 


The  above  results  in  the  following  two  latin  squares  of  order  4 
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6 

Q 

P 

V 

P 

a 

6 

a 

6 

\ 

P 

p 

6 

a 

.  HI 

II 

IV 

I 

.  IV 

I 

III 

II 

I 

IV 

II 

fiT 

II 

III 

I 

IV 

The  above  method  of  constructing  mutually  orthogonal  latin  squares  using 
polynomial  coefficients  works  for  latin  squares  of  order  n  where  n  =  2  .  We 
need  another  procedure  for  other  valuea  of  n  and  shall  now  construct  a  latin 
square  of  order  6  from  the  orthogonal  polynomial  coefficients  in  the  table  of 
single  degree  of  freedom  contrasts  fo,r  36  combinations.  If  we  observe  only 


the  signs  of  contrasts  we  note^/hat  the  36  combinations  may  be  classified  into 
six  sets  of  four  with  like  signs  and  two  additional  sets  of  six.  The  latter  two 

I 

sets  will  be  used  to  build  up  the  six  sets  of  four  into  six  sets  oi  six  as  follows 
where  all  combinations  with  a  plus  sign  go  in  the  one  level  and  all  those  with  a 
minus  sign  go  in  the  2ero  level: 

(R,C2»r  (k3C3,i'  (R4C4)0’  (R5C5,0  *  2  fr0m  (R1C1>1»  (R2C2,l<R3CJ)r  (R4("'4V 

,R2<Vo1  (R3CVa’  fR4C4,l’  1  “ 

,R2C<2,0’  ‘ R 3 C 3 'b ’  lR4C4)0’  (R5C5!0  ‘ 

^2^2*0’  V  (R4<“'4>0’  ^R5<''5,1  +  2  ^rorI1  ^l^l^O ’  *R2^2*1  ’ ^3^3*0 ’ *R4^4*1  ‘ '  R5','6 *0 

(R2C2’o’  ,R3C3,r  (R4C4>1’  (R5C5,0* 
lR2C2)l’  lR3C3i0’  (R4C4)0’(R5C5,1  f 


{ rom  these  sets  we  obtain 


i 


(.12  +  21  +  34  4  43)  4  (00  4  55)  =  A 

(02  4  204  35  4  53)  4  (11  4  44)  =  B 

(01  4  10  445  4  54)  4  (22  4  33)  =  C 

(04  4  15  +40  4  51)  4  (23  f  32)  =  D 

(03  4  25  +30  4  52)  4  (14  +  41)  =  E 

(13  4  24  +31  4  42)  +  (05  +  50)  =  F 


This  results  in  the  following  latin  square  of  order  6s 


00 

T" 

10 

~c" 

20 

~ 

30 

40 

D 

50 

r 

01 

c 

11 

B 

21 

A 

31 

F 

41 

L 

51 

D 

02 

B 

12 

A 

22 

C 

32 

D 

42 

r 

52 

E 

03 

I 

13  F 

23 

D 

33 

C 

43 

A 

53 

B 

04 

~d" 

14 

- 

24 

l" 

34 

A 

44 

B 

54 

C 

05 

T~ 

15 

D 

25 

C 

35 

13 

45 

C 

55 

A 

The  pair  of  treatments  in  the  second  set  of  parentheses,  e.g,  (00  +  55),  were 
picked  from  the  set  of.  six  in  such  a  manner  as  to  have  i  and  j  in  the  combina¬ 
tion  ij,.  contain  0,  1,2,  3,  4,  and  5  since  each  letter  mu9t  appear  once  in 
each  row  and  once  in  each  column. 

It  would  be  interesting  and  perhaps  enlightening  to  carry  out  the  above 


procedure  for  n  *=  10  aind  12  and  to  exhaustively  study  the  complete  set  of  35 
contrasts  for  n  *  6  . 


i 


04  .!-> 
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V.  Group  Construction  of  Q(n,t)  Sets 


V.  0.  Introduction.  The  construction  of  0(n,t)  sets  based  on  groups  and  their 
associated  mappings  such  as  automorphism,  complete  mapping,  and  orthomorphism 
is  the  oldest  and  still  the  most  popular  method  for  n  not  of  the  form  4t  +  2  , 

Euler  [1782]  implicitly  utilized  some  properties  of  finite  groups  of  order  2t  4  1 

and  ^t  for  his  construction  of  0(2t  +  l,2)  and  Q(4t,2)  sets,  respectively.  It 

V  f. 

was  MacNeish  [1922]  who,  for  the  first  time,  explicitly  (however,  hot  rigorously) 

utilized  group  properties  for  his  construction  of  0(qITl,qrn-l)  sets  and  Ofn,  V  ) 

{1  ‘i  1  r 

sots,  where  q  is  a  prime,  m  is  a  positive  integer  and  If  n  »  q,q,  ...  q 

/,  1  /  r 

12  r 

is  the  prime  power  decomposition  of  n  then  K  =  min(q^  ,  •»•••>  qf  J  .  The 

field  construction  of  0(qm,qm-l)  sets  found  independently  by  BoBe  [19  38] 
and  Stevens  11939]  is  based  on  the  additive  group  of  GF(qm)  and  its  related 
cyclic  group  of  automqrphisms.  The  Otn,n-i)  sots  for  n  -  3, 4, 5, 7, 8  and  9 
..exhibited  by  Fisher  and  Yates  [1957]  are  based  on  cyclic  group  and  abelian  groups. 
Several  beautiful  applications  of  group  theory  to  the  existence  and  non-existorco 
of  0(n,t)  sets  have  been  found  by  Mann  [1942,  1943,  1944]  .  The  0(12,5)  sets 
found  by  Johnson  ot_  al,  [  1961]  and  Bose  et  al.  [I960]  are  based  on  abelian 
groups  of  order  12.  Hedayat  [1969]  and  Hedayat  and  Federer  [1969]  have  found 
a  series  of  results  on  the  existence  and  non-existence  of  0(n,t)  sets  through  the 
group  theory  approach.  The  interested  reader  on  this  subject  will  find  the  fol¬ 
lowing  references  together  with  the  references  given  to  these  papers  very  useful: 
Page  [1951]  ,  Page-Hall  [  1955]  ,  Singer  [1961]  ,  Bruck  [1951]  ,  and  Sade  11956]  . 


/ 
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The  author  has  no  doubt  that  the  reader  can  find  many  more  interesting  papers 
directly  or  indirectly  related  this  rich  subject. 

V.  1.  Definitions  and  Notations. 

There  are  several  forms  of  definitions  of  latin  squares  and  orthogonal 
latin  squares.  The  following  forms  are  useful  for  the  results  whioh  will  follow: 
Definition  V.  1. 1.  A  latin  square  of  order  n  on  an  n«set  £  is  an  n  x  n  matrix 
whose  rows  and  columns  are  each  a  permutation  of  the  set  ]£.  Every  latin  square 
of  order  n  may  therefore  be  identified  with  a  set  of  n  permutations  (p^p^, ...  ,Pn> 
where  pt  is  the  permutation  associated  with  the  ith  row. 

Definition  V.  1.  2  .  Let  Lt  be  a  latin  square  of  order  n  on  an  n-set  ^ "» 
i  =  1, 2, . . ,  ,t  .  Then,  the  set  S  *  { L^,  Lv  .  ..  ,  Lt)  is  said  to  he  a  mutually 
orthogonal  set  of,  t  latin  squares  if  the  projection  of  the  superimposed  form  of 
the  t  latin  squares  on  any  two  n-sots  ^  and  l  +  i,  t$rm$  a  permutation 

of  the  cartesian,  produdt  set  of  ^  ancl  •  Such  a  set  is  denoted  as  an 
*0(n,t)  set. 

Definition  V.  1.  3  .  If  s  (P^,  P^,  ’ '  *  1  ^ln^  ^2  *  ^21*  ^22 '  ‘ '  *  *  ^2n^ 

are  two  latin  squares  of  order  n  on  an  n-sot  then  we  may  define  l1L2  t0 

be  L,  =  (PnPit*  P,,P,3 . P,  P,  >  .  The  generalization  to  the  product  of. 

3  11  21  12  22  in  2n 

t  >2  latin  squares  follows  immediately. 

V.  2 . 2.  Construction  of  Q(n.t)  Sets  Based  on  a  Group: 

We  shall  divide  the  problem  into  three  parts  based  on  whether  n  is  a 
prime,  or  a  mixture  of  prime  powers.  The  proof  of  the  subsequent  results  can  be 
found  in  the  references  related  to  this  section. 
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V*  EL  s  fl  a  prime  .  Recall  that  any  prime  ordered  group  is  cyclic. 

Theorem  V.  1. 1. 1.  Let  G  *  {P,  ,P, . P  }  be  a  cyclic  permutation  group  of  de 

i  c  q 

flree,  g  and  order  g  .  Then,  ■  (L^  L,.  ^  is  an  Q(q,q-1)  set, 

where  ^  =  (pj,  p* . p*)  . 


tratlon 


1,  Let  q  s  5  ,  Seiect  any  arbitrary  generator  such  as 


,13  3  4  5, 


(3  5  2  14^  which  generates  a  cyclic  permutation  group  G  and,  hence,  a  iatin 
square  Lj  .  Then, 


For  those  who  do  not  like  to  work  with  permutation  groups  we  present  the  following 
theorem: 

Theorem  V.  2. 1.  2.  Ls*t  L(r)  be. an  n  x  n  square  with  ri  +  j  (mod  q)  in  its  u,  nth 
£Sl).  1,J  ■  0,1,,,,  , q — 1  ,  Then .  SltJ  -  {L(i>,  LU) . Liq-l)}  Is  an  Otq.q-ll 


set  if.  q  is  a  prime. 


Let  q  *  5;  then, 


II 


Note  that  L(  l  >  in  theorem  V.  2. 1.  2  is  based  on  the  cyclic  permutation 
group  generated  by  and  Mil  .  l‘<1>,  1.2,3,...,,-!,  H«"o. 

theorem  V.  2, 1.  2  is  a  special  case  of  theorem  V.  2. 1. 1  , 

V.  2,  2.  n  =  qm  whei£  q  Is  a  prime  and  m  any  Positive  integer-  Note  that 

*  i 

this  case  in  particular  for  m  =  1  includes  case  1.  We  shall  present  three 
theorems  for  this  case.  The  first  two  are  based  on  cyclic  groups  and  the  third 
one  is  based  on  any  group  which  admits  an  automorphism  of  order  t  . 


Theorem  V.  2,  2. 1.  let  G  =  {P  ,  P2, 
degree  n  and  order  n  .  Then.  S 

M  1 

where  n  ®  qm  \  »  q  -  i  , 


V 

8  <L1’L2 


be  a  cyclic.permutatlon 
}  is  an  Oln(l 


flrcj>up  of 


)  set 


\ 

Demon stratloh  V.  2 .  2. 1.  Let  n  ■  3  a  9  .  Select  any  arbitrary  generator  such 
A  l  3  4  5  6  7  8  9,  tl  . 

3  4  5  1  6  7  S  9  2  which  «*nerates  a  cyclic  permutation  group  G  and  hence 


a  latin  square  L  .  Then,  since  \ 


U 

u 

u 

a 

u 

a 

a 

H 

H 

5 

n 

a 

a 

B 

a 

a 

fl 

a 

□ 

D 

a 

a 

a 

a 

B 

fl 

n 

& 

a 

a 

a 

a 

Q 

a 

fl 

B 

1 

D 

a 

B 

B 

D 

B 

B 

B 

5 

a 

m 

a 

D 

D 

B 

B 

B 

Q 

H| 

a 

a 

B 

B 

B 

a 

B 

Q 

B 

a 

□ 

o 

□ 

B 

B 

B 

hi 

a 

o 

B 

a 

B 

B 

□ 

a 

B 

B 

B 

B 

a 

H 

Bfl 

a 

B 

B 

a 

B 

B 

B 

D 

B 

B 

a 

B 

B 

B 

□ 

B 

B 

B 

a 

B 

B 

B 

B 

B 

B 

D 

a 

B 

0 

B 

a 

B 

a 

□ 

a 

□ 

B 

B 

B 

a 

B 

□ 

a 

B 

a 

B 

a 

B 

B 

a 

D 

a 

a 

a 

B 

B 

a 

B 

B 

B 

a 

a 

Dl 

B 

B 

B 

B 
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a 
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is  an  0(9,2)  set. 

Conjecture.  The  set  S2j  is  orthogonally  locked,  meaning  that  there  does  not 
exist  a  latin  square  L*  such  that  S21  U  {I,*}  is  an  0(n,X+l>  set  if  n  is  not 


[i] 


a  prime.  Note  that  for  n  even  this  conjecture  is  correct  since  any  latin  square 
of  even  order  based  on  cyclic  permutation  group  is  orthogonally  mateless. 


An  analogous  theorem  to  theorem  V.  2. 1.  2  for  this  case  is: 

Theorem  V.  2.  2.  2.  Let  L(r)  be  an  n  x  n  square  with  ri  4  j  (mod  n)  in  its 
(i,j)  cell,  i  *  0,1,2,. . .  ,n-l  .  Then  =  {L(l),  L(2),  . . .  ,  L(  X  )}  is  an 
0(n,X)  set  If  n  =  qm  and  X  =  q-1  . 

2 

Demonstration  V.  2.  2.  2.  Let  n  *  q  *  3  then, 
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and  L(2)  * 
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is  an  0(9,2)  set.  Note  that  theorem  V.  2.  2.  2  is  a  special  case  of  theorem  V.  2 . 1 . 1 
viz.  ,  L(l)  is  based  on  the  cyclic  permutation  group  generated  by  i®  .  \  ' ' '  n  i 

1  W  J|||  U 

i  .  / 

and  L(i)  ®  L  (1),  1  «  2, , , ,  ,X  . 

Theorem  V,  2.  2,  3.  Let  G  *  =  e  the  identity,  a^ ,  . . ,  ,  a  j  be  a  .group  oj 

Q-rdor  n  ajid  o  anaijlt_q.ffi2r£ll^l-9l SL4SL  t  on  G  .  then, 

1)  S  »  {Lj,  L^,  . . .  ,  L^}  is  an  0 < n ,  t )  sot,  where 

\ 
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* 


Li=  *(a3 


,  2 , . .  ■  , 


i*ia» 


0(n,  n-1)  sat  from  the 
la»t  n-1  row. 


tey  latln  «c 


e 

MX) 

ff2(X) 

•*(x) 

o(;<)  , 

<»(x)a(x) 

a(X)v2(X) 

sea 

a(x)ot(x) 

o2(x) 

'  2,  . 

<*  <X)« 

2  2 

Ur  (X)»  (X) 

a  a  a 

e2(X)et(x) 

a  (x)a(x) 


»l(x)e2(x)  ... 


crSxJar'fx) 


for,  am,  x  ia  g  axaaat  the  idsntlw  element. 

We  see,  therefore,  that  by  means  of  theorem  V.  2.2.  3  one  can  construct 
an  0(n,t)  set  if  we  can  find  a  group  G  and  an  automorphism  a  of  order  t  . 

In  particular,  if  t  ■  n-  1  the  whole  task  of  construotlon  reduces  to  the  construc¬ 


tion  of  L.  as  described  above.  If  n  *  q  then  because  every  elementary 
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abelian  q-group  G  of  order  n  admits  an  automorphism  a  of  order  n-1,  we 
can  construct  an  G(qm,qm-i)  set  uased  on  G  and  a  .  Here  we  present  * 
general  method  of  constructing  such  an  automorphism  for  any  n  =  qm  .  In  par¬ 
ticular,  we  shall  exhibit  such  an  automorphism  for  the  following  n  : 

*  n  »  2^  n  «  2,  3 . 9 

n  ■  3m,  m  «  2,  3,  ...  ,  6 
n  1  5m,  m  a  2,  3,  4 

n  »  7m,  m  =  2,  3 

2  2  2  2  2  2  2 
y  n  a  11  ,13  ,17  ,  19  ,23  ,  29  ,  and  31  . 

This  will  then  perhaps  be  the  largest  table  that  has  ever  been  produced  so  far 

for  G(n,n-li  sets, 

Note  that  there  la  no  loss  of  generality  if  we  limit  ourselves  to  the  follow¬ 
ing  elementary  abelian  q-group  of  order  n  »  qm  . 

G  ■  {(bj  b.,  , .  ■  b^),  bj  =  0,1,2,...,  q-1,  J  3  1,  2,  , , . ,  m}  . 

>;< 

The  binary  operation  on  G  is  addition  mod  q  componentwise,  viz.,  b2  . . . 

b  )  •* .  (b|  b'  . , ,  b'  )  '■  (c,  c,  ...  c  i  where  c  =  b.  +  b!  (mod  qi  .  Note  that 
m  i  c  .  m  i  c  m  1  1  1 

the  elements  of  G  are  simply  the  treatment  combinations  of  m  factors  each 
at  q  levels.  .The  reason  why  we  have  chosen  this  particular  elementary  abelian 

q-group  Is  that  it  has  a  well-known  structure  to  those  who  are  concerned  with  ex- 

>!« 

periment  design  construction.  Note  also  that  G  is  the  direct  product  of  m 
Galois  fields,  each  of  order  q  . 

The  generator  set  for  every  elementary  abelian  q-group  of  order  qm  con¬ 
sists  of  m  elements,  and  for  uniformity,  we  may  choose  the  following  ordered 


i 
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generator  set  for  G  . 

g  =  {(100  ...  0),  (01,00  ...  0),  ...  (00  . ..,  010),  (00  ...  01)}  . 

I 
i  ' 

I  Note  that  the  structure  of  every  automorphism  a .  on  G  Is  completely 
defined  if  we  know  the  image  of  each  element  of  g  under  a  .  G  is  a  vector 
space  of  dimension  m  over  GF(q). 

Before  proceeding  further  we  need  the  following  well-known  result] 
Theorem  V.  i.  Z.  4.  Let  G  be  an  elementary  abelian  q-gr_oup  of.  order  n  *  qm  . 
Then.  Auto  G  is  isomorphic  to  the  (multiplicative)  croup  of  all  non-singular 
m  X  m  matrices  with  entries  In  the  field  of  integers  mod  q  . 

$ 

The  construction  of  an  automorphism  of  order  n-1  for  G  is  equivalent 

n-1  t 

to  the  construction  of  an  m  x  m  matrix  A  such  that  A  *  I  but  A  #  I  if  t 
is  not  a  multiple  of  n-1  over  the  field  of  integers  mod  q  .  * 

We  know  from  linear  algebra  that  if  $  is  a  linear  map  on  a  vector  space 
V  and  if  x  »  V  such  that  x*  0  but  o<x>  *  x,  then  1  is  an  eigenvalue  of  *. 

'  '  \  ''  S  8 

Moreover,  if  {k^k, . \t) 'is  the  set  of  eigenvalues  .of  then  1  {kj ,  \ 

s  s  * 

. . .  ,  K  t  )  is  the  set  of  eigenvalues  of  <r>  .  Therefore,  for  our  problem  we  must 

find  a  linear  mop  on  c"  with  a  set  of  eigenvalues  \  having  thp  property  that 

for  each  i,  k  ®  *  1  (mod  q)  for  all  s  *  1,  l, . . ,  ,n-2  and  k  ^  *  *  1  ,  To  do  so 

let  T  be  a  Gr(qm)  and  let  A  be  a  generator  of  the  multiplicative  cyclic  group 

of  Gr!qm),  i.e.  p1  *  1,  i  =  1.2, . . .  ,n-2  while  |3n  *  =  1  .  Let  f(x)  be  a 

memo  irreducible  polynomial  over  GF'fq)  for  (3.  Note  that  f(x)  has  degree  m 

A  la  sometimes  called  a  primitive  root  or  mark  of  F  .  New,  if  we  let  A  be  the 

companion  matrix  for  >3,  then  it  is  easy  to  see  that  A  has  the  desired  property. 


Example  ? 

let  us  findan  automorphism  of  order  i  for  G  =  {(00),  (01),  (10),  (11)} 
It  is  sufficient,  by  previous  arguments,  to  find  a  2  x  2  matrix  A  of  order  3 
over  the  field  of  integer  mod  2  .  Let  GF(2^)  =  (0,  1,  (3,  (3  +  1}  with  the  follow¬ 
ing  addition  (  +  )  and  multiplication  (•  )  tables 


BDD 

mm 


0  0  0  0 


3  (3  h  1 


m 

■  0 

pH  1 

m 

|i  -  1 

0 

P  +  1' 

n 

p 

>  u 

Note  that  p  is  a  primitive  root  for  Gl  i2  i  and  fix)  *  x“  +  x  +  1  is  a  monic 


irreducible  polynomial  for  p,  since  f(p)  e  0(mod 2)  .  The  companion  matrix 


associated  with  f(x)  is 


As  a  check 

.2 


sk  - 
0 


n  .1 

i  l 


over  GH2),  A  * 

10  0  1 


over  GF(2i  .  ■ 

Let  us  now  determine  the  image  of  the  ordered  generator  set  g  =  {(10),  (01)} 


0  1  (10)  0  (10)  +  1  (01) 


*m\i  JL(oiu  =  L1  ■■ 


+  i  (01)  (01) 

4  i  (01)  =  01) 

J  U 


Therefore,  A(10)  *  (01),  A(01)  -  (11),  and  since  (11)  =  (10)  +  (0!),  (00)  *  2(10) 

4  2(011,  we  have  A(ll)  a  (10),  A(00)  a  (60)  , 

S! 

Now,  we  have  a  group  G  1  of  order  4  and  an  autoftojrphiem  of  order  3 
on  G  .  We  can  now  construct  an  0(4,3)  set  .  Since  e  a  (00),  and  If  we  let 
x  =  (10)  in  theorem  V.  2.2.  3,  we  obtain: 


■  A(10) 


A2  (10) 


A3(10) 


AilO)  A<10>  /  A(10)  A  (10)  A(10)  A  (10) 


A^  (10 )  A2(10,A(10|/  A2(10)A2(10)  A2(10)A3(10) 


A3,i0)  /  A3(10IA(10)  A3(10)A2(10)  |  A"(10)A'(10) 


(10) 


The  other  two  latin  squares  are  obtained  by  a  cyclic  permutation  of  the  last  three 
rows  of  L.  .  Tnus, 


(00) 

(01) 

(in 

(10j 

(ioi 

(in 

(01  ) 

(00) 

(01) 

(00) 

(10) 

(It) 

(ii) 

( 10) 

(00) 

(01 ) 

(01) 

(11) 

(10) 

(11) 

(10) 

(00) 

(01)’ 

(10) 

(11) 

(01) 

(00) 

(01) 

1 

(00) 

(10) 

(11) 

To  simplify  the  notation  we  set  (00)  =  1,  (01)  =  Z,  (11)  -  3,  (10)  =  4  to  obtain 


3  2  1 


,  and  L  = 


3  2  1 


We  are  now  ready  to  exhibit  a  generating  matrix  of  order  n-  l '«  q 
with  entries  from  GF(q)  for  those  n  promised  before. 


Generator 


Order 


0  10  0 
0  0  10 
0  0  0  1 
110  0 


0  1  0  0  0  0 

0  0  10  0  0 
U  0  0  1  0  0 

0  0  0  0  10 

0  0  0  0  0  1 
l  10  0  0  0 


0  1  0  0  0  0  0  o 
0  0  1  0  0  0  0  0 
0  0  0  1  0  0  0  0 
0  0  0  0  1  0  0  0 
0  0  0  0  0  l  0  0 
0  0  0  0  0  0  10 
0  0  0  0  0  0  0  1 
10  11  10  0  0 


Genera  tor 

"0  1  0 
0  0  1 

h  o  lj  , 

To'  ioo  o 
0  0  10  0. 
o  o  o  i  n 
0  0  0  0  1 
10  10  0 


o..  1,0  0  0  o.  0 
o  o  i  o  o  bo 
o  0  0  10  0  0 
0  [)  0  0  1  0  0 
0  0  0  0  0  10 
i  1  0.0  0  0  0 


0 / 1  0  0  0  0  0  0  0 

oo  i  n  o  o  o  o  b 

ooo  ;  ooooo 

0  0  0  0  1  'o  0  0  0 

000001000 
0000  0  0  100 
000  0  00010 
000000001 
111110  0  0  1 


Generator 


Order 


n 


Gen  erator 


Order 


5  4 


■  3 
7 


.7* 


0  1  0 

0  0  1 

1  1  0 

Mb 


~0  l  0  0 

"o  1  0  0  0 

0  0  10 

'  80  1 

35 

0  0  10  0 

0  0  0  1 

,0  ,0  0  10 

'i.  i  o  o 

0  0  0  0  1 

1  10  00 

0  10  0  0  ? 

0  0  l  0  0  0 

0  0  0  10  0 

0  0  0  0  l  0 

0  0  0  0  0  1 

1  1  0  0  0  0 


728 


0  1  0 
0  0  1 
2  2  0 


0  1 
1  1 


0  1  0  0 
0  0  10 
0  0  0  1 
3.3  0  3 

>  M 


024 


I 


0.1  0 
0  0  1 

2  2  0 


342 


26 


242 


124 

48 

120 

168 

360 

728 

960 


To  shed  more  light  on  the  given  procedure  we  go  through  another  exampie.  Let 


- - 


G '  =  {(000),  (001),  (010),  (Oil),  (100),  (101),  (110),  (111)} 


V  *  {(100),  (010),  (001)}  and 


A  = 


0  i  u 
0  0  1 
1  0  1 


0  1  0 

(100) 

(010) 

Ag  a 

,001 

(010) 

a 

(001) 

1  0  1 

(001) 

Um  mm. 

(101) 

mm  •m 

Let  x  in  theorem  V. 2.  2.  3  be  ( IQO)  Then, 


A(100)  *  (010)  , 
A2(100)=  (00  1)  , 
A3(100)=  <  101 1  .v, 
A4(100)=  till)  , 
A5(100)=  (110)  , 

A6(100) -  foil)  ,  and 

7  V  ' 

A  (100)=  (100)  . 

Therefore,  we  obtain  as  follows: 
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(000) 

Setting  (000)  =  1,  (010)  =  2,  (001)  =  3,  (101)  =  4,  (111)*  5,  (110)=  6  , 
(  0  1 1 )  =  7,  (100)  a  8,  then  Lg  in  a  compact  form  will  be: 
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B 

a 

B 

Now,  we  can  derive  Lj,  L, . Lf)  from  LQ  by  a  cyclic  permutation  of  the 

last  7  rows  of  L  ,  for  example, 
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fl 
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fl 
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fl 
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b 
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D 
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fl 
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fl 
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3 

B 

fl 

8 

fl 
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2 

B 
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3 

8 

8 

T 

7 
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D 

2 
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5 

7 

fl 

DI 

2 

and  so  on.  Note  the  way  Lj  is  derived  from  LQ  s  except  for  the  first  row  of 


1'0  J!'d  which  are  identical,  the  ith  row  of  Lfl  becomes  the  (i4l)th  row 


t>f  Lj,  and  the  last  row  uf  LQ  becomes  the  second  row  of  .  In  general  Lj 


is  derived  from  L  _j  in  the  same  fashion  as  Lj  is  derived  from  l,  . 


m^  m, 

V.  2.  3.  n  =  qj  q2  . . .  qf  ,  where  qi  is  a  prime  such  that  qi  #  if  i  # 

and  m  is  a  positive  integer,  1  3  1,2 . r  . 


m 


mi  m2 


lilggasa  V.  2f  3, 1.  Let  n  =  ,  q^ . _qfr 

of  n  .  then,  there  exists  an  0(n,V  set  base; 


mpoaltlon 

m, 


^  2  m 

^2  » *  * • » )“f  • 


where  y  =  mint'qj  , 


Construction.  Let  n(  *  .  Then,  by  the  method  of  theorem  V.  2.  2.  3  construct 

an  Oftij,  n|- i)  set  3^  »  (L^,  L12,  ,  Lin  _r>,  i  *  1,2,. .,,r  .  Now,  let 

3  ( Lji i fL^2 »  •  •  •  i  Lj^}  *  iai,2,,.tfr  .  Then,  H  *  {Aj,  Ag,  •  •  •  ,  A  }  is 
an  C(n,  \)  fl0t  where  A^  *  L^  ®  L2j  ®. . .  ®  denotes  the  Kronecker  product 

operation. 

Demonstration  V.  2.  3.1.  Let  n  *  12  =  22  •  3  Then,  y«  2, 


12  3 


12  3 


si*  Ln  ■  LlIjJ  1  »  Li2  ■  31  2  , 


3  1  2 


2  3  1 


L21  “ 


12  3  4 

TTTT 
2  j |_TT 

4  3  2  1 


L22 


T 

T 

T 

T 

4 

3 

2 

"i 

2 

1 

T 

7 

3 

4 

1 

2 

a— 

|  1  2  3  4 

TTTT 

TTTT 

2  14  3 


S1  "  ^LU'  L12^  ftnd  S2  12i  ^L21’ L22^  *  Til®n»  the  reader  can  easily 
verify  that 

H  3  tAi  *  Lu$  L21«  A2  ■  L12®  L22  } 

is  an  0(12,  2)  set. 

Remark.  Let  n  and  y  be  the  same  bb  in  theorem  V.  2.  3. 1.  Then  It  can  be  shown 

i 

that  automorphism  method  fails  to  produce  more  than  y  mutually  orthogonal  latin 
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squares.  We  shortly  show  that  this  inherent  defect  is  due  to  the  mapping  not 
ic  the  yiuup  structure, 

■RsIiPlUgfl  v»  Consider  for  each  positive  integer  n  an  abstract  group  G  of 
order  n  with  binary  operation  *  .  Let  n  be  the  collection  of  all  one-to-oni 
mappings  of  G  into  itself.  Then  two  maps  *  and  ^  in  U  are  said  to  be 
orthogonal  if  for  any  g  ~  in !  G  . . .  .  '  . .  ' 

(<r  Z)  =:•  fvp Z) 

has  a  unique  solution  Z  in  G,  In  particular  if  «•  is  an  identity  map  then  4, 
is  said  to  be  an  orthomorphlam  map.  A  t-subset  of  fl  is  said  to  be  a  mutually 
orthogonal  set  if  every  two  maps  in  this  t-subset  are  orthogonal. 

Let  L(* )  be  an  n  x  n  square.  We  make  a  one-to-one  correspondence 
between  the  rows  of  L<* )  and  the  elements  of  G.  Thus,  by  row  3c  we  shall 
mean  the  row  corresponding  to  the  element  x  in  G  .  Similarly  we  make  a  one- 
to-one  correspondence  between  the  columns  of  L(* )  and  the  elements  of  G  . 

The  cell  of  LM  which  occurs  in  the  intersection  of  row  x  and  column  y  is 
called  the  cell  (x,y).  , 

Ih.e,9rem  V,  4,  3,  2r  J^t  <r  Dejr)  H  .  Put  in  the  cell  (x,y)  2^  L(* )  the  element 
<rr  x)  y  of  G,  Gall  the  resulting  square  L(<r )  .  Then  L(*  )  is  a  la  tin  aau*™ 
n  2Q  G  .  Maaaver  if  Uj,  t2>  ....  «rt>  la  a  set  of  t  mutually  ortho¬ 
gonal  maps  then  {Lio^) . L(<r  ))  Is  an  0(n,t)  set. 

P.?fT1!?..n.5,^ti9n  vi  ^1  3 >  Let  G  *  { 0 ,  i,  2}  with  the  binary  operation  x,  +  x,  ■ 

A  H 

Xjtmod  3),  Xj  in  G  .  Then  the  maps  o-  and  41  with  the  following  definitions 
are  orthogonal. 
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(T  (0  )  =  0 
«•{!)=! 
i r{2)  »  2 

The  corresponding  latin  squares  to 


4-(0)  *  0 
i.n  \  -  •» 

rt  1  '  —  ** 

4(2)  =  I  . 

» 

<r  and  4i  are: 


0 

1 

z 

1 

2 

Q 

2 

0 

1 

1*4)  * 


0 

l 

2 

2 

0 

1 

1 

2 

0 

which  are  orthogonal. 


V.  3.  Construction  of  0(n,t)  seta  baaed  on  t  different  oroupa  of  order  n 

Up  tolnow  we  have  been  concerned  with  the  construction  of  0(n,t)  sets 
using  a  group  ;of  order  n  which  admits  certain  mappings.  In  this  section  we  want 
to  show  that  for  some  n's  and  t's  one  can  construct  0(n,t)  sets  based  on  t 
different  groups  each  of  ordor  n  .  This  approaoh  proved  useful  because  it  lead 
to  the  construction  of  an  0(15,  3)  set.  We  should  mention  that  our  motivation 
to  search  along  these  lines  has  stemmed  from  the  following  theorem,  with  a  nega¬ 
tive  flavor,  proved  by  Mann  [1944]  , 

Theorem  V.  3. 1.  \ ILltiiMfilllblaJte. fianilmgLftn  0(5,2)  set  based  on  two  dlf- 

\ 

toajjBgrauiaiM  flight. 

Tor  a  while  we  thought  that  this  theorem  might  be  true  for  other  orders. 
However,  it  was  found  that,  fortunately,  this  is  not  the  case  as  the  following  two 
theorems  show: 

Theorem  V.  3. 2.  It  is  possible  to  construct  0(7,2)  seta  based  on  two  different 
cyclic  permutation  groups  of  order  7  . 
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Proof:  By  construction  {Lj,  L£>  is  an  0(7,2)  set  where 
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L,  and  L,  are  based  bn  two  different  permutation  groups  as  can  easily  be  seen 
1  *  I 

from  the  different  structure  of  their  rows.  To  be  specific  is  based  on  the 

1234567 

v  cyclic  permutation  grot^p  generated  by  <37^,  j  4  2  5 1  anc*  **2  based  on  the 
cyclic  permutation  group  generated  by  (3  j  4  $  *  7  •  Note  that,  since  Lj  and 

are  based  on  cyclici  permutation  groups,  then  by  theorem  V.  2. 1.1  {L^}  and 
{L^}  can  be  embedded  llh  0(7,6)  sets.  However,  whether  or  not  {Lj,  oan 
be  embedded  in  a  larger  set  is  an  open  problem. 

Theorem  V.  3.  3,  It  is  possible  to  construct  0(15,  3)  S£iaJmS&BIL  Um  dif¬ 
ferent  cyclic  permutation  groups  of  order  15. 

Wc?  remind  the  reader  that  every  group  of  order  15  is  cyclic. 

Proof:  By  construction  {Lj,  ,  Lj  I  is  an  0(15,  3)  set  where 
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l  11  10  7  9  14  13  t'  0  3  4  2  8  5  12/ 


and 
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Whether  or  not  L.^,  Lj}  can  be 'embedded  in  an  .0(15, t ),  t  >  3,  set 


is  an  open  problem. 


V.  4.  Concluding  Remark 


Johnson  ot  eC.  [19olJ  and  Bose  etal.  |i^bOJ  independently  found,  by  an 

1  -•  *  1  •“  .  i 

electronic  computer,  five  mutually^rthogonal  latin  squares  by  first  finding  five 
mutually  orthogonal  maps  for  an  abol'ian  group  of  order  12.  The  0(12,  5)  set 
exhibited  below  is  the  set  found  by-  Johnson  et  all  [1961]  .  Note  t]iat  the  top 
square  is  obtained,  after  a  proper  renaming,  as  the  direct  product  of  a  latin  square 
of  order  2  and  a  cyclic  latin  square1  of  order  ft  being  both  orthogonally  mateless. 
Moreover,  every  other  square  is-ubtainod  oy  proper  row  permutations,  determined 
by  an  orthomorphism,  from  the  top  square. 


final  Komark.  The  group  method  falls  to  produce  an  0(n,t)  set,  t  >  2  for  any 

n  i-l  ih* ■  :  ..rrr,  4t  <  2  .  This  is  so  because  the  Cayley  table  of  any  group  of  order 
n  -  at  >  2,  which  is  a  latin  square  of  order  n,  is  orthogonally  mateless. 
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VI.  Projecting  Diagonals  Construction  of  O(n.t)  Sets 

A  very  simple  procedure  (sort  of  the  "man-on-the- street"  approach)  of 

constructing  balanced  incomplete  block  and  partially  balanced  Incomplete  block 
2 

designs  for  v  =  k  j  items  in  incomplete  blocks  of  size  k  hds  been  utilized 
since  the  'late  if  4(>y  by  the  author  and  has  its  oountarpart  in  constructing  0{n,t) 
sets.  First  we  sftatti  Illustrate  its  use  in  incomplete  block  experiment  design 

i 

construction,  and  thfen  wt>  show  how  it  applies  to  the  construction  on  0(n,t)  set. 

Th"  theoretical  basis  for  tnis  method  may  be  derived  directly  from  the  preceding 
section.  ,  ■ 

/’  '  .  .  .  ■  !  ■ 

The  procedure  becomes  apparent  through  an  example.  Suppose  that  v  =  9 

/ 

and  k  =  3  .  After  writing  thfe  first  square  as  illustrated  below,  take  successive  ( 
diagonals  of  the  preceding  square  and  use  them  to  form  the  incomplete  blooks  of 
a  square,  thusi  r- 


Square  1 


4 

S 

■ 

7 

8 

g 

Square  2 


l 

5' 

9 

2 

6 

7 

3 

4 

8 

Square  3 

H  6  8 


Square  4 


2 

4 

9 

3 

5 

7 

1 

4 

7 

2 

5 

8 

T 

~6~ 

T 

As  we  have  noted  this  is  a  resolvable  balanced  incomplete  design  with  the  para¬ 
meters  v  ~  9  *-  k  s  3,  r  *  1  *  k  O,  b  *  12  *  k(k+l),  and  X  =  1,  where 

*  -** 

th<  r  ws  A  the  above  squares  form  the  incomplete  blocks. 

2 

Tv  form  a  partially  balanced  incomplete  block  design  for  v  ■  k  in  in-, 
complete  blocks  of  size  k  one  may  use  any  2,  any  3,  . .. ,  any  k  arrangements 
i or  squares  i.  To  illustrate  the  formation  of  a  partially  balanced  Incomplete  block 
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I 


i. 

C 


i  i 
!  1 


r 

i 


design  for  v  =  6  *  k(k-l),  r  =  2-or  3»k,  and  k'  «  2  simjly  delete  the 
numbers  7,8,  and  9  from  the  last  k  »  3  arrangements.  The  deletion  of  certain 
symbols  from  the  set  1,2,,..  ,v  is  known  as  "variety  cutting".  For  k2  =  25 
and  k  »  5  partially  balanced  incomplete'  block  designs  may  be  constructed  for 
v  a  10  and  k  *  2,  v  *  15  and  k  =  3,  and  v  *  20  and  k*  «  4  by  the  above 
"variety  cutting"  procedure. 

Also,  the  successive  diagonals  method  is  useful  for  v  =  k2  in  incomplete. 

'  - ]  '  ,  v 

blocks  of  si'^e  k  for  any  odd  k  .  For  example,  for  v  =  225  and  k  =  15  four 
arrangements  or  squares  may  be  quickly  constructed  by  the  above  method.  Like- 

I  ,  N 

wise,  the  "variety  cutting"  procedure  may  be  utilized  to  obtain  2  or  3  arrange¬ 
ment's  for  v=  15p,  2  <p<_15,  varieties. 

The  above  method  has  Its  counterpart  in  constructing  mutually  orthogonal 
latin  squares  and  this  possibility  is  briefly  mentioned  in  Fisher  and.Yates  ( 19 5 7  J 
in  this  context,  AgaU.the  method  becomes  apparent  through  an  example.  First 
write  the  latin  square  in  standard  order  and  of  the  form  given  below  for  the  first 
square,  then  project  the  main  right  diagonal  of  the  preceding  square  into  the  first 
Column  of  a  square,  and  then  write  the  symbol  order  in  the  same  manner  as  in  the 
first  square.  As  a  first  example,  let  the  order  of  the  lat,ln  square  be  3;  the 
squares  are: 


I 

j 

i 

) 

i 


first  square 


T 

2 

3 

2 

3 

1 

3 

1 

2 

second  square 


~T~ 

2 

3 

3 

1 

2 

2 

3 

1 

third  square 


1 

2 

3 

1 

2 

3 

1 

2 

3 
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Thus,  the  mein  right  diagonal  of  the  first  square  is  1,3,2  which  becomes  the 
fliai  column  or  the  second  square.  Then,  write  the  first  row  as  1,2,3,  the 
second  row  as  3,1,2,  and  the  third  row  as  2,3,1  .  For  the  third  square, 
which  is  not  a  latin  square,  the  right  main  diagonal  of  the  second  square  is 
1,1,1  and  this  becomes  the  first  column  of  the  third  square;  the  rows  are  then 
completed.  If  we  then  take  the  right  main  diagonal  of  the  third  square,  we  ob¬ 
tain  the  first  square. 

As  a  second  illustrative  example,  the  five  squares  for  order  n  =  5  which 
are,  constructed  by  successively  projecting  diagonals,  arei 


first  square 
i  1 2, 1  3  4  V 

TTTT7 

3  4  5  12 

T7T7T 

TTTT* 


second  square 

v _ _ 

J  I  2  I  3  I  4  I  5 

TTTTT 
T”  TTT 

tt  5  5  r 
TT  i  i  r 


third  square 
1  2 1  3  1 4  1 5 

TTTTT 

TTTTT 

TTTTT 

TTTTT 


fourth,  square- 

1  2  3[  4|  5~ 

T  1  2,  3  4 

T  5  1  2  3 

TTTTT" 

2  3  4  5  1 


fifth  square 
l  (  2|  3 1 4 1 6 

TTTTT 

TTTTT 

TTTTT 

12  3  4  5 


The  ;Tfth  square  is  not  a  iatin  square  out  may  be  utilized  to  construct  the  first 
square  through  use  of  the  method  of  successive  projections  of  the  main  diagonals. 

The  method  may  dc  utilized  for  any  odd  order  n,  and  will  produce  qj-  1 
orthogonal  latin  squares  for  n  =  q^,.q^. . .  .  ,qs  where  q^  <  q1+j  and  q^,q^ , . . .  ,qg 


is  the  prime  power  decomposition  of  n  .  Thus,  for  n  =  15  »  3(5)  a  pair 

(qj-1  *  3-1  a  2)  of  orthogonal  latin  squares  is  easily  produced.  For  n  =  35 

/ 

5(7),  a  quartet  of  mutually  orthogonal  latin  squares  is  readily  produced  by  the 

/ 

/  • 

projecting  diagonals  method/ 

/  '  .  \ 


I 


3 
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VII.  Relating  Between  Complete  Confounding  and  Simple  Orthomorphlsms 


We  shall  illustrate  the  ideas  by  going  through  a  complete  example  taking 

l 

n  =  1 2  =  2  X  3  .  For  this  purpose  we  take  the  ring  of  12,  elements  (obtained 
by  utilising  Raktoe's  ( 19 69 J  results)  as  follows: 

- . •  -  -  -  sriiu  1  •  h 

i  0  0 

1  *6  3 
x  3x 

x+1  3X4 3 

R^  =  lit:  *4  3  3,4,5,  3x ,  3x4l,  3x42,  3x'+3,  3x44,  3x45} 

is  a.  commutative  ring  under  addition  and  multiplication  (moddd6, 

■  2 

3x  -»  3x  +  3*'  4x  +  4)  in  the  following  sBnsc: 

e.g.  !  (a).  (3X43)  +  (3x44)  *  hx  4  7  *  1;  here-we  hajve  to  reduce 

j  . 

only  mod  6  to  get  the  answer. 

(b).  ( 3x+l)  ■  (3x-t  4 )  =  'ix*  4  15x  4  4  3 

-  3x2  *  3x  J  4  ! 

-  i  340]  X2  4  1  340]  X  4  [044]  ■  (3x2  4  3x)  4  4' 

-  3  4i  =  i;  here  first  we  had  to  reduce 

mod  6,  then  mod  ix2  4  3x  4  3  leaving  us  immediately  3  and 
4,  which  is  irreducible  mod  4x  4  4,  thus  resulting  in  1  , 


Explicitly,  to  facilitate  arithmetic,  tfe^eaddition  and  mutliplldation  of  these  12 
elements  are: 


+ 

0 

1 

2 

3 

4 

5 

3x 

3X41 

3x42 

3x43 

3x44 

3x45 

0 

0 

1 

2 

3 

4 

5 

3x 

3X41 

3x42 

3x4  3 

3x44 

3x45 

1 

2 

3 

4 

5 

0 

3X4  1 

3X42 

3x43 

3x44 

3X45 

3x 

2 

"4 

3 

0 

1 

3x42 

3x4  3 

3X44 

3x45 

3x 

3X41 

3 

.... 

0  ' 

1  , 

2- 

.3X43 

3X44 

_3x4£L 

__  3x 

3X4l 

3X42 

4 

1 

2 

3 

3X44 

3X45 

3x 

3X41 

3x42 

3X4  3 

5 

i 

4 

3X4  5 

3x 

3X41 

3x42 

3x4  3 

^X4  4 

3x 

0 

L  ' 

2 

3 

4 

5 

3x+i 

2 

3 

4 

5 

0 

3x42 

v, 

4 

5 

0 

1 

3xt  3 

■  0 

1 

2 

3x44 

i 

2 

3 

3x45 

' 

1 

4 

• 

0 

1 

2 

■  3 

4 

5 

1 

3x 

3X4  1 

3X42 

*• 

3X4  3 

3x4  4 

1.  r 

3x+5 

o ! 

0 

0 

0 

0 

0 

0 

0 

0  ‘ 

0 

0 

0 

0 

1; 

1 

2 

3 

4 

5 

3x 

3X41 

3x4  2 

3X4  3 

3x44 

3x45 

2|- 

■-  4 

0 

2 

4 

0. 

2 

4  , 

0. 

2 

4 

3 

3 

0 

'  3 

3x 

3X43 

3x 

3x4  3 

3x 

3X4  3 

4 

4 

2 

■  0 

4 

2 

0 

4 

2 

5 

' 

1 

3x 

3x45 

3X44 

3X4  3 

3X42 

3X4  1 

3x 

3X43 

3 

3x4  3 

3 

3X43 

3 

3X+1 

3x44 

5 

3x 

'  1 

3X42 

3x42 

3X41 

3 

3X45 

1 

3X43 

3X4  3 

3 

3x 

3x44 

3X41 

5 

3X45 

3x44 
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Now,  associate  with  a  la  tin  square  of  order  12  the 


32x42  *  [3X  4]  X  [3X  4] 


--  U  x  1 2  lattice  square  with  the  following  breakdown  of  the  143  degrees  of 


freedom : 


A4 

2 

c3 

3' 

B4 

2 

D3 

3 

aV 

2 

3  3 

G-D^ . 

■  3  ■ 

A4B* 

2 

cVx 

3 

V _ 

c3d3x+3 

3  j 

r 

6 

„4„3 

B  C 

6 

4  4  3 

A  B  C 

6 

a4b2c3 

6 

a4d3 

b 

B4_3 

B  D 

6 

4  4  3 

A  B  D 

6 

a4b2d3  . 

aVd3 

6 

b4o3d! 

6 

.4433  - 

A  B  C  D 

\6 

a4b2c3d3 

6 

aVd3x 

6 

bVd3x 

b 

a4b4c3d3x 

6 

a4b2c3d3x 

6 

aW*  +  3 

(■> 

b4c3d3x43 

b 

a4b4c3d3x+3 

6 

a4b2c3d3x+3 

6 

For  any  row  or  column  confounding  we  need  to  confound  effects  totaling  up  to1,.  11 
degrees  of  freedom.  There  are  natural  candidates  available.  In  fact,  we  may 
choose  for  our  first  lattice  square  the  confounding  scheme. in  many  ways.  A 

scheme  resulting  in  a  pair-of  orthogonal  latin  squares  is  the  followingj 

\ 
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Confounding  scheme  of  our  12  x  12  lattice  square 


Treatments  identified  with  A  B  C  O  I 


Using  the  complete  confounding  approach  as  outlined  above,  one  can  construct 
min  [  (22-i t,  (i-i)j  3  i  mutually  orthogonal  latin  squares  and  no  more  as  can 
easily  be  observed  from  tlie  degrees  of  freedom  table. 

From  the  multiplication  table  of  our  ring  R.,,  we  observe  that  1,  5,  3x+l,  / 

■  ,  '  _ j' 

3x+2,  3x+4  and  3x+5  are  the  6  non-zero  divisors  (i.  e.  elements  with  multiplica¬ 
tive  inverses).  Following  Bose,  Chakravarti  and  Knuth  [1960]  ,  we  consider 

simple  automorphisms  in  R,,  of  the  forms  * 

12  { 

t  *  .  v 

a<ri  -  r  r 

whj?re  r  is  a  given  fixed  element  having  a  multiplicative  inverse  (because  only 

»  •  :  •  ’ 

these  elements  are  capable  of  producing  automorphisms  of  Rj^)  ,  Let  now  oulr 
aim  be  to  produce  two  orthomorphisms  which  in  turn  will  produce  an  0(12,  2) 
set.  For  this  purpose  consider  the  automorphisms!!,  ; 

l 

liri  =  r  '  j 

*  ■  u(r  i  *  r  •  r  . 

•  ’  ■  ■  |  '  ' 

>•  >jt  | 

Now  I* -a  implies  tho  condition  that  r  in  the  equation  [r  •  r-rj  ■  c  ha*  a 

unique  solution  for  every  c  of  R  .  In  our  setting  this  means  that  r(r  -  1)  =  a 

"■  *  u  , 

0 

has  a  unique  solution,  i.e.  ,  r (r  +  5)  =  c  has  a  unique  solution  which  in  turn 

"  i  -l  ' 

implies  that  (r  *  5i  .  exists  In  R^  . 

Substituting  in  the  values  of  r  we  see  that: 

(1  4  5j  *  does  not  exist  in  R^ 

M  it  ii  i, 

((3x4-1)  4  5J  ^  does  not  exist  In 

* 
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[(3x42)  +5]  *  =  [3x41]'  *  exists  in  fr. 


12 


-1 


12 


[(3x44)  4£]  does  not  exist  in-  R 

*  ^ 

[(3X45)4  5]  *  =  [  3x44]  *  ekists  in  R 


12  ‘ 


Hence  we  have  obtained  two  fialrs  of  orthomorphisms  namely* 


I(r)  *  r 


o^(r)  =  (3x+2)r 


.  and 


l(r)  *  r 


“ilf)  =  (3x45  )r  . 


w*  * 

The  0(12,  2)  set  present, ecKabove  using  complete  confounding  corresponds  to 


/.the  first  pair  of  maps.  It  may  be  easily  shown  that  simple  maps  of  the  type 


<>  i 


o(r)  *  r  •  r  lead  to  0(12,  2)  seta  or  in  general  to  an1  0(n,  a)  set,  where 

n  *  m  M  V-  L»  M 


a  -  min 


nl  n2  '  nk  JL .  n  . 

(p.  -1,  p,  -1,  ...  ,4  -r.1)  and  n  -  TT,  p,**  so' that  the  cpmpleto 

-  i  .  •  »  ■  1*1^  \ 


'con- ; 


X. 


\ 


'founding  approach  is  equivalent  to  the  cfonstruc^iorwof  a  set  of  a  orthomorphisms . 


.o 


\  4 

\ 

A 


■■  \ 


V 


\  .. 
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e  Remarks  on  "( mthor  . ..rphism  “  Construction  of  Oln,t)  Sots 


i;,  i  uu  l.  i.  I'.uker  .snowed  by  a  combination  of  classifica¬ 

tion  of  oaoor.  ,".v::n  oonoiicra die  eliridu-.iUuu  uf  i&oinuiphic  repetition  accessible 
to  cat  down  on  computer  time,  foliowea  oy  computer  runs)  that  there  are  obtain¬ 
able  only  5  orthogonal  laun  squares  of  order  12,  all  restricted  to  be  copies  of 

p  wixfc“latTn^qu3?ei^^  permutation 8 . 

(Tho  researchers  cited  call  this  the  method  of  orthomorphisms.  Parker  considers 
this  J,  n./  only  oast  :  ::  freaks  of  luck;  further,  Patkcr  feels  that 

wT t ti-w pr» ii ,  u— mils  fiu  pteciiM..-  uef  m  it  ion  •  i 

Parker  rr.adc  another  finding,  also  by  hard  classification  of  cases  followed 
by  computer  runs,  which  Marshall  Hail  feels  is  more  important  than  that  cited 
aacve.  \k,  pair  vf  :roer-J2  rrtr.-i  gorcal  squares  of  the  type  mentioned  can  be  ex¬ 
tended  to  a  complete  set  ut  ary  sort;  ;.e.  ,  further  orthogonal  la  tin  squares  are 
allowed  to  be  completely  general. 

■.Vhat  right  oo  obtainable  for  crtncgonul  squares  of  order  20  in  like 

:  i 

far-r.iv re  .v*- pc  .-muring  ‘.ha  no-r.-c,  clic  Abelian  group  of  order  20  is  an  interest¬ 
ing  matter  for  speculation  —  cor.ceivaoly  one  might  even  produce  a  complete  set 
•  pi  jrth-.g'.r.ai  squares  equivalent  to  a  plane,  Knuth  and  Parker  discussed  the 
problem  ir.  P'  ),  and  cone  leu- -I  th  it  exhaustive  search  is  out  of  the  question; 
■mil  ;  -  r* a : . - : t * :  -icpl-  -  f  -an--,  .-.cm:  pr '-au-r  art  attractive  result. 

,r.  i ic.-sva  at  th"  r  //-perr.utaucn  i "orthomorphism “  of  Bose 
an.j  -  h  r.  •  ippr  a  nr  for  gmn  of  order  15,  and  proved  by  Hasse- 

Mmn'  a* nr. ,  ir-.viri  th.v  a  t ;-.!--te  ;mt  could  net  p*->  so  obtained.  He  dropped 

1 40 


farther  v/vik;  but  ‘.one  pers ist**r.c-  •  roui'i  quit"  p<  ssiuiy  yit  i  J  ^ ' r r .  ub  fiv* 
orthogonal  square.;  of  order  15  . 


;ria  jtluif.  or.  e>ru-_*r  15  or  £•,  .m  ignt  oe  unv^rtticon  oy  an  ur  oitio 


investigator.  (The  facts  for  order  id  mentioned  above  rule  out  charces  here,  i 
-One-- might  produce  sots  of  orthogonal  latm- squares -of  r&w^pern.uteGhgroup  .type , 


using  automorphisms  of  the  group  laim  squares  to  eliminate  —  or,  that  failing, 
'•educe  —  isomorphic  repetition.  It  would  not  be  shrewd  to  program  a  computer 
pr.  ^c“  a;i  t:  ».n.-.v>.-i --:1s  .f  >  group  la  tin  square,  for  running  Usr.<-  and  output 
•would  be  excessive*;  then  for  any  hint  of  efficiency  it  would  be  necessary  to  tor: 
uir.-u*  and  do  a  reduction  on  the  computer  output..  After  a  set  of  row-perrr.ute  u 
latin  squares  (possibly  exhaustive  for  oroer  15,  but  almost  certainly  only  -a 
sample  for  order  Jo  >  large-  '••rough  mat  computer  searching  would  require 
realistic  amounts  of  time,  or."  might  proceed  with  the-  next  step.  Produce  ail 
trans'vdrsals  of  the  set  of  orthogonal  squares  by  computer,  then  fit  these  tegeth* 
i rv 1  e» j  L  possible  ways  ‘again  by  -*<»n  outer i  tc  form  orthogonal-  mates  of  the  f-r ■  ■ 
t  of  thogonu i  i.itir,  squares.  Unlike  Parker's  assertion  aoove  aoout  c.  r.pi- 
-•  11:  order- in  ?quar«  s  .  there  it;  r.  ;  ciewn  argument  implying  impossibility  : 


producing  H  .rntnuyo;.*!  la  fir.  -:q.--ir.*s  t  order  Is-  th 


r.  •*  l>l  l 'Cl  .'i  tt  *•  *•» . 
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'•  is  well-*:n  #n  that  rpnovmg  one  line  from  the  plane,  usually  called 


the  line  of  infinity.  the  remaining  n^'  +  n  lines  can  be  arranged  into  2n  lines 
passing  through  two  points  at  infinity  which  are  arbitrary  up  to  notation  and 
eoordinatiaation  of  the  plane,  and  n2~n  .  lines  belonging  to  n>  1  mutually 
orthogonal  latin  squares.  If  the  line  at  infinity  is  chosen  to  be  a  secant  and 

there  are  dr  lir.es.  the  lines  pass  through  the  two  points  of  the  oval  such 

that  each  of  the  n-  1  latm  squares  consists  of  ~  secants  and  ^  non- 

intersectors  passing  through  each  of  the  n-  1  points  at  infinity  other  than  the 
points  of  the  oval.  ' 

'•  7.’ sing'  the  described  method,  it  was  assumed  that  a  plane  of  order  10 
exists,  y.r:  jer’t.tis  a 3 sum. t ton  dl  lines  could  he  exniDited  arbitrarily  up  to 
notation.  '  of  tnese  lines  ore  was  taken  to  bn  th°  line  of  infinity  and  the 
remaining  2r=  used  to  non  ■  :r uuz°  the  plan0.  Then  oy  trial  and  error  twenty 
'.ir.'-s  were  f:  .mu  wm  o  hr  r~'-  :  tv/-,  orthogonal  latir.  squares.  The  method 
USt.-.  -irsT  ;ct  squares  differs  from  the  one  described  in  literature. 
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Unfortunately  no  more  squares  could  :<♦*  Ivund  .iSir.q  this  ■  ",  '  t;.-;  ■< 

computer  established  that  the  two  squares  did  not  yield  an  additional  multi  ill 

orthogonal  mate.  Clearly  it  could  happen  that  the  choice  of  the  first  two  was 
unfortunate.'  The  same  method  was  applied  to  the  plane  of  order  i<i  .  Here 
the  trial  and  error  method  failed  to  produce  even  two  orthogonal  squares.  St 

may  be  worthwhile  to  remark  that  the  construction  of  the  plane  and  cons'-quert 

the  search  for  orthogonal  latin  squares  dees  not  require  the  assumption  that 
the  oval  consists  of  the  maximum  number  of  points  r,  *  1  .  However,  if  the 
plane  does  not  include  an  oval  consisting  of  n  *  i  points  the  lines  could  nc. 
be  classified  into  two  categories  only  and  this  complicates  the  construction  • 
the  plane.  Let  us  illustrate  the  mei.nod  in  the  case  n  equals  in  .  I:  is 

easy  to  show  that  in  this  case  the  oval  must  consist  oi  at  least  (<  point;-  . 

However,  the  case  of  an  oval  of  t  points  would  be  ignored  sine-  m  this 

case  every  quadrangle  would  have  to  have  ccllir.ear  diagonals.  On  tn-  it  he; 
hand,  a  plane  of  order  ten  must  be  a  non  -  Des  a  rg  u  e  s  s  la  n  and  hence  .must  m  n: 
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a  n  : 


Suppose  that  'hr 


!»■■!*  r‘‘fdt*  '5  i.i  !;ur  ;lc  wit  ollioeiir  diagonals. 


i  lar.e  c..r.tdut!s  a  qua  Jiar-.glo  vith  u.x.  nullinear  diagonals  and  suppose  that  the 

plane  contains  an  ovai  consisting  of  seven  points  then  the  104  points  of  the 

plane  wr  :h  do  not  belong  to  tha  oval  could  be  classified  Into  three  categories: 
t i >  points  lying  on  3  secants.  1  tangent  7  nonintersectors 

i  li  I  "  "  2  i  6 

<1111  "  1  3  3  " 

Let  us  r.atr.e  the  number  of  points  in  each  category  by  x.  y,  2  respectively. 
Clearly  x  +  y  *  z  -  104  . 

Counting  the  intersections  at  the  secants  and  the  tangents  we  get  the 

* 

further  equations: 

3x  •  /  -  103 

J.y  *  10z  •  32S 

The  unique  solutions  of  this  system,  of  equations  are  x  -  20,  y  -  43,  z  =  39  . 
Ore  coulci  start  the  construction  <,(  the  plane  under  the  present  assumption  and 

investigate  the  pos sioilit ies  of  wntaimng  orthogonal  latin  squares  in  this  way. 
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X.  Code  Construction  of 


;(n,t)  .>ets 


Given  an  n-symbol  alphabet.  e.g.  ,  1.2 . n,  anu  a  set  ci  n-upi- 

of  the  n  symbols,  we  denote  the  set  cl  ail  k- tuples  by  C.  .  This  set  may 

k ,  n 

be  thought  of  as  a  vector  space  or  as  a  k-dimensicr,al  hypercube  with  edges  of 
length  n  ...  Any  subset  of  ^  is  denoted  as  a  block  code  with  a  block  length 
of  k  .  The  elements  of  the  subset  are  denoted  as  code  words.  The  number  of  ’  • 
symbols  by  which  any  two  code  words  differ  is  called  the  Hamming  distance. 

If  any  pair  of  code  words  in  the  subset  differs  by  a  Hamming  difference  c:  v. 
least  r.  the  block  code  is  called  a  distance  r  code  .  A  distance  r  ... 

called  an  (r-l)/2-error  correcting  code  because  fewer  than  ir-li/2  changes 
leaves  the  word  closer  to  its  original  form  than  to  any  other  code  word  in  the 
subset.  For  similar  reasons,  a  distance  r  code  has  also  been  designate u  as 


an  <r-l>- error- dectectlno  code. 


In  an  interesting  paper,  Golomb  and  Posner  j  1964]  discuss  the  roianon- 

2 

ships  between  a  subset  of  n  code  words  and  an  Om.ti  set  and  :elat<-  t(v  -■ 
to  ideas  developed  from  a  consideration  of  a  set  of  super  rooks  of  : 
on  the  r.t+^  chessboard  such  that  no  two  super  rooks  attack  each  other.  V:\< 


new  concepts  of  rook  domains  and  took  packing  were  found  to  be  vnr.. 


providing  a  geometrical  view  of  the  results. 

2 

Any  Subset  of  n  words  from  C,  which  forms  a  s  ingle-error-d*- 1  u 

3 ,  n 

code  may  be  used  to  construct  a  latin  square  of  order  n  as  any  pair  oi  ’r." 

i 

triples  diifers  by  at  least  two  symbols.  Likewise,  any  subset  of  r,‘  w-;.r 
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from  ,  with  a  Hamming  distance  of  t  +  1  may  be  utilized  to  construct 
an  On  ?i  set.  These  results  are  embodied  in  the  follnwtnn  theorem  (from 
Golomb  and  Posner  [1964]  >: 

Theorem  X, 8.1  The  following  three  concepts  are  equivalent; 

1}  jn  0<n,t)  .set 

ID  A  set  of  n2  nonattaoklng  super  rooks, oj.j?o.Mter  t  on  the  nt42  board. 

2 

Tor  even  t.  also  the  following,  a  set  of  n  super  rooks  of  power  t/2  on  the 

t*  2 

n  board  such  that  no  cell  is  attacked  twice;  that  is.  such  that  the  rook 
domains  are  r.onoverlapping. 

2 

Hi)  A  distance  t  +  1  code  of  block  length  t  +  2  with  n  words  from  an 
n-svmbol  alphabet. 

Tor  those  interested  in  code  construction,  reference  may  be  made  to 

Mann  |19h8]  and  Peterson  [19M]  and  the  literature  citations  therein.  We  shall 

merely  illustrate  the  method  of  construction  of  an  0(n,t)  set  from  n2  words 

of  longtn  t  *  2  and  Hamming  distance  t  ■+  1  through  an  example.  Let  n  -  1 

1 

and  t  2  .  Then  tne  n*"  -  q  code  words  with  length  4  and  Hamming  distance 
1  and  the  corresponding  latin  squares  are: 


latin  squares  of  order  3 
u  1  2  0  12 


1)000 

on: 

0222 

n 

0 

1 

2 

0 

0 

1 

2 

10  12 

1  12!) 

1201 

to  produce 

1 

1 

2 

0 

and 

1 

2 

0 

1 

2021 

2  102 

22  10 

2 

2 

0 

1 

2 

1 

2 

0 

where  tine  first  symbol  corresponds  to  row  number,  the  second  to  column  number, 
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the  third  in  symbols  in  the  first  latin  square,  and  the  fourth  to  symbols  in  the 
second  latin  square.  The  two  latin  squares  form  an  Jf3.2?  set.  Note  that  any 
pair  of  the  quadruples  differs  in  at  least  three  symbols. 

The  analogy  of  the  above  with  many  of  the  concepts  from  fractional  repli¬ 
cation  and  orthogonal  arrays  is  Immediately  apparent.  The  equivalences  of  many 
of  the  results  in  these  fields  need  to  be  systematically  noted  much  in  the  same 

manner  that  Golomb  and  Posner  1 19 64 ]  note  various  equivalences  among  Om,ti 

l  t»2 

sets,  error-correcting  codes ,  and  n  nonattacking  rooks  on  an  n  chess¬ 


board 


XL  Pairwise  Balanced  Design  Construction  of  0(n,t)  Sets 


Central  to  the  constructions  of  orthogonal  latin  squares  of  Bose  and 
Shrlkhande  (1959)  and  of  Parker  (1959,  1960]  is  the  following  which  might  be 
called  a  "folk  theorem,"  being  credited  to  no  specific  investigator:  From  a  set 
oft  orthogonal  latin  squares  of  order  n  one  may  produce  a  set  of  n2  ordered  J 
jt*2i-tuples  on  n2  symbols  such  that  each  pair  of  distinct  positions  contains 
each  ordered  pair  of  symbols  (exactly  once);  the  converse  construction  can  also 
bo  carried  out.  iSome,  such  as  Bose,  prefer  to  call  the  set  of  ( t-f 2 J- tuples  ar, 
orthogonal  array,  i  There  is  nothing  difficult  to  prove  in  this  construction.  Two 
arbitrary  positions  in  the  itfii'tuplus  are  identified-with  row  and  column  indices 
in  matrices,  and  each  other  position  with  entries  in  one  of  the  matrices.  The 
equivalence  between  orthogonality  of  latin  squares  and  the  conditions  on  the 
it- 2  (-tuples  is  then  fairly  apparent. 

Paricer  [I960]  contributed  the  following  to  the  construction  of  orthogonal 
la  nr.  squares,  if  tnore  exists  a  pair  of  orthogonal  latin  squares  of  order  m,  then 
they  exists  a  pair  of  orthogonal  latin  squares  of  order  3m  +  1  . 

Lot  the  3m  -  1  symbols  be  X,...,,X  and  :he  residue  classes  <mod  2m +1). 

1  m 

r or.-,  the  i latin  square)  array 

X  ■.  i  -i 

0  X  - 1  i 

1 

*  -i  X  0 

1 

-i  i  0  X  . 

1 


\ 


i  cr  eac.'i  i,  1  <  j  <  m,  each  row  is  one  of  tn<-  ordered  quadruples.  la  turn,  the 
list  quadruples  is  built  up  by  adding  each  integer  (mod  dm  *  ll  to  all  t.,ur 
positions  at  once,  the  X.  symbols  being  unchanged  by  the  addition,  ine  set 
of  4nu2m  -1  1)  ordered  quadruples  just  described  contains  in  each  pair  of  distinct 
positions  exactly  one  occurrence  of  each  ordered  pair  made  up  of  ag  an<4  a 
t<3Sidue  class  in  either  order,  and  of  each  ordered  pair  made  up  or  two  distinct 
residue  classes.  The  required  set  of  ordered  quadruples  is  completed  by  adjoin¬ 
ing:  n  all  ordered  quadruples  (j,j,j,j),  j  =  0,  . . .  ,  Zm;  iii  a  set  of  ordered 

quadruples  on  the  Xj  symbols  corresponding  to  a  pair  of  orthogonal  latin  squares 

V.IP 

of  order  m  guaranteed  by  the  hypothesis  to  exist. 

t 

Bose  and  Shrikhande  (1959,  published  1959  and  i960  partly  in  a  3-author 

t 

paper  with  Parker)  developed  a  sequence  of  constructive  theorems  which  led  in 
steps  to  disproof  of  Euler's  conjecture  for  all  orders  4t  •»  <d  >  6  .  Their  central 
theorem  given  here  does  not  exhaust  their  methods,  but  virtually  all  their  results 
rest  on  this  theorem.  We  begin  withd  definition.  A  pairwise  balanced  design. 

P 3 ( n ;  kj . k^l  is  a  collection  of  subsets  of  a  set  of  n  elements,  each  sub¬ 

set  having  number  of  elements  one  of  the  k  ,  and  such  that  each  pair  of  distinct 
elements  in  the  set  of  n  occurs  in  a  unique  subset  of  the  PB  .  (Note:  unlike 
m  balanced  incomplete  block  designs,  the  subsets  of  a  PB  are  not  restricted  to 
nave  equal  numbers  of  elements,  i  Now  for  the  main  theorem  of  Bose  and  Shrikhandi 
If  a  PBtn;  kj, .  . .  ,k  )  exists,  and  for  each  i,  1  <_  i  <_  t,  a  set  of  m  orthogonal 
latin  squares  of  order  k^  exists,  then  a  set  of  m  -  1  orthogonal  latin  squares  of 
order  n  exists.  Loosely  speaking,  the  sets  of  ordered  tuples  for  each  subset 


?tf  ‘ 


t 


of  the  PB  are  constructed  and  these  fit  together  to  form  a  set  of  ordered  tuples 
fur  the  full  set  of.  n  elements.  The  decrease  from  m  to  m-1  orthogonal  latin 
squares  occurs  because  in  fitting  the  pieces  together  to  form  the  large  set  of 
ordered  tuples  ,  it  Is  necessary  that  each  set  of  ordered  tuples  formed  from  a 


subsei  of  the.  PB  .includer-each  if i ^  i y '  dvef  'the  elements 
of  that  subset.  (It  is  sufficient  that  this  condition  be  fulfilled  in  the  construc¬ 
tion.  Thus  the  theorem  might  be  stated  in  slightly  stronger  form:  "H  .  . .  l<_i<_t 
a  set  ol  m  orthogonal  latin  squares  of  order  k.  with  a  transversal  exists,  then 
a  set  of  m  orthogonal  latin  squares  of  order  n  exists.  ")  Now  for  a  more  nearly 

formal  version  of  tnc  proof.  If  there  exists  a  set  of  m  orthogonal  latin  squares 

-  2 
of  order  n,  then  there  exists  a  set  of  the  appropriate  sort  of  n  ordered 

im->  l)-tuples  with  each  symbol  repeated  in  an  (m+l)-tuple  m-^l  times.  (The 

condition  mentioned  is  satisfied  with  <m-»2 (-tuples  if  the  set  of  orthogonal  latin 

squares  has  a  transversal.  )  One  need  simply  put  together  the  ordered  tuples  on 

each  subset  of  the  PB  in  turn,  subject  to  the  important,  condition  that  within 

each  subset  of  the  PB,  each  tuple  of  repetitions  of  each  symbol  be  included. 

Carrying  this  out  on  the  alphabet  of  the  symbols  in  each  sunset  of  the  PB,  one 

has  the  construction  for  the  set  of  orthogonal  latin  squares  in  the  conclusion; 

‘vich  ordered  tuple  of  a  repeated  symbol  among  the  n  is  used  only  once. 

A  representative  and  very  interesting  example  (Bose  and  Shrlkhande  in¬ 
formed  Parker  that  this  was  the  first  case  of  disproof  of  Euler's  conjecture  pro¬ 
duced  in  their  joint  work  at  a  blackboard  )  yields  5  mutually  orthogonal  latin 


squares  of  order  50  via  the  PB  construction.  One  forms  the  affine  plane  of 


order  ?,  then  adjoins  exactly  on'*  ideal  point  on  each  line  of  one  class  of  paral¬ 
lel  lines.  This  yields  a  PB(50;  8,7).  Since  there  exist  6  orthogonal  latin 
squares  of  each  order  8  and  7,  there  exist  6-1  =  5  orthogonal  latin  squares 
of  order  50  . 

a:4fJh:rt^tWn  oh-  the=8osasShtilchande„:Rfi:  co.nstructlphi:  MideJtrdfn 
trivial  PB  designs,  having  a  single  subset  of  all  elements,  any  PB  has  a  sub¬ 
set  with  at  most  one  more  element  than  the  square  root  of  the  number  of  elements 
in  the  large  set.  Thus  other  techniques  are  requisite  to  produce  more  than  \n 
orthogonal  latin  squares  of  order  not  a  prime-power. 


XII.  Product  Composition  of  Q(n,t>  Sets 


About  ~0  years  ago,  for  the  first  time,  Tarry  [1899]  in  his  half-page  note 
asserted  that  if  there  exists  an  0(a,2)  set  and  if  there  exists  an  0(b,  2)  set 
then  there  exists  an  0(ab,2)  set.  Ho  exhibited  the  following  0(12,2)  set,  by 
composingtwoG(3,2)andO(4v2)  set5,  to  demonstfatf;TheW 
tion.  Note  that  in  the  following  square  the  set  of  first  integers  belong  to  one 
latin  square  and  the  set  of  second  integers  belong  to  the  second  latin  square. 

Mu  more  description  is  given  by  Tarry. 
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Tarry  did  not  observe  any  generalization  of  his  method.  Perhaps  this  was 
due  to  the  fact  that  he,  like  so  many  other  researchers,  was  only  concerned  with 
sets  of  type  0(n,  2)  .  Probably  ho  was  not  aware  of  the  existence  of  a  larger  set. 
About  23  years  later  MacNeish  f  1922]  demonstrated: 
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1)  The  existence  and  a  construction  of  an  0{n,  n-1)  set  for  n  a  prime  or 
prime  power  integer. 

2>  A  generalization  of  Tarry's  procedure  viz.  ,  if  there  exists  an  0(a,  r)  set  and 

if  there  exists  an  0(b, r)  set  then  there  exists  an  C(ab,r),  set. 

i)  By  a  succ«s#ive  application  of  t*  and  2)  be  showed  that  if:  n  -  p,  p*  • » , »  P;  = 


is  the  prime-power  decomposition  of .  n  .then  there  exists  an  0(n,r)  set  where 

a 

r  =  min  {p.  -  1,  i  =  1,2, . .  .  ,t  }  . 

MacNeish  co^ld  not  embec  ms  0(n,  r)  set  generated  in  3)  in  a  larger 
set.  This  unsuccessful  attempt,  reinforced  by  Euler's  conjecture,  led  MacNeish 
to  prove  (erroneously)  geometrically  that  0(n,t)  sets  do  not  exist  for  t  >  r  , 
and  therefore,  as  a  confirmation  of  Euler's  conjecture.  The  preceding  argument 
of  MacNeish  is  known  as  MacNeish's  conjecture  in  the  literature.  By  constructing 
an  0(21,  3)  set  Parker  [  19 59 J  gave  a  counter  example  to  MacNeish's  conjecture. 
l>ater  Bose,  Shrikhande,  and  Parker  [1960]  completely  demolished  Euler's  con¬ 
jecture  except  for  n  =  fc  .  It  should  be  mentioned  that  MacNeish's  conjecture 
has  not  been  totally  disproved  yet.  Tor  instance,  no  one  as  yet  as  far  as  we  know 
has  constructed  an  0(15,4)  set  (an  0(15,3)  set  is  given  in  section  V  for  the 
first  time)  or  an  0(20,3)  set.  We  believe  that  MacNeish  should  be  given  sub¬ 
stantial  credit  for  his  non-erroneous  contributions.  It  is  to  be  regretted  that 
MacNeish  is  often  cited  in  the  literature  only  for  his  false  conjecture. 

Even  though  Tarry  and  MacNeish  did  not  attach  any  name  to  their  pro¬ 
cedure,  it  is  not  difficult  to  see  that  it  is  the  method  of  Kronecker  product  of 


matrices.  Therefore,  we  can  state,  more  formally,  their  results  as  follows! 


Theorem  (Tarry- MacNeish). 


If.  {A^,  A2,  . . . ,  A  }  Is  an  O(n.r)  set  and  if 
[S^,  S2,  . .  .  ,  b  f  IS  an  0(m,rj  set,  then  {AJ  $Bj,  A2  <g>  B2>  . . .  ,  Af  ®  Br  }  , 
where  $  denotes  the  Kronecker  product  operation  of  matrices.  Is  an  0(nm,  r) 


3  Crt , 

The  preceding  arguments  clearly  support  the  choice  of  the  title  for  this 
section  andisin;cohtrast  to  the  choice  of  the  name  for  the  procedure  given  in 


section  XIII. 


'  A 
J  .  •« 
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XIII.  Sum  Composition  Construction  of  O(n.t)  Sets 


XIII .  1.  Introduction.  Perhaps  one  of  the  most  useful  techniques  for  the  con¬ 
struction  of  combinatorial  systems  is  the  method  of  composition.  To  mention 
some,  here  are  few  well-known  examples:  1)  If  there  exists  a  set  of  t  ortho¬ 


gonal  latin  squares  of  order  and  if  there  exists  a  set  of  t  orthogonal  latin 

squares  of  order  n2,  then  there  exists  a  set  of  t  orthogonal  latin 

squares  oi  order  n^n^  .  2)  If  there  are  Steiner  triple  systems  of  order  and 


v  ,  there  is  a  Steiner  ‘nple  system  of  order  v 


3)  If  and  K  are 


two  Hadamard  matrices  of  order  n^  and  respectively,  then  the  Kronecker 
product  of  Hj  and  is  a  Hadamard  matrix  of  order  .  4)  If  Room 

squares  of  order  nj  and  exist,  then  a  Room  square  of  order  n^n^  exists. 
=>)  If  BIB  'v  ,  k ,  \  ^ )  and  BIB  (v,  k,  \  exist  and  if  f(t  2v^)>  k,  then 


BIB 


<vlV2,k,KlV2)  ex-sts  where  f( V  ^ )  denotes  the  maximum  number  of  con¬ 


straints  which  are  possible  in  an  orthogonal  array  of^size  V  2v2,  with  v2  levels, 
strength  2,  and  index  .  b)  As  a  final  example,  the  existence  oi  orthogonal 
arrays  ( K  |v*,  q  v  ,  t),  1=  1,2,...  ,r  implies  the  existence  of  the  orthogonal 
array  (kv,q,v,t),  where  k  =  v  •>•  •  • v  r>  v  B  W“Vr'‘  and  q  =  mln(q1'  V 
.  . .  ,qr>  • 


The  reader  will  note  that  each  of  the  above  examples  involved  a  product 
type  composition.  The  method  that  wn  will  describe  utilizes  a  sum  type  compo¬ 
sition,  by  means  of  which  one  can  possibly  construct  sets  of  orthogonal  latin 
squares  for  all  n  >  10  . 

XIII.  2.  Definitions  .  In  the  sequel  by  an  0(n,t)  set  we  mean  a  set  of  t  mutu 
ally  orthogonal  latin  squares  of  order  n  • 

i  '  -C' 


a)  A  transversal  (directrix}  of  a  latin  square  L  of  order  n  on  an  n-set  2 


i  c  a  "'tier,  cf 


entries  uf  these  ceils  exhaust  the  set 


2.  and  every  row  and  column  of  f  represented  in  this  collection.  Two  trans¬ 
versals  are  said  to  be  parallel  if  they  have  no  cell  in  common, 
b)  A  collection  of  n  cells  la  said  to  form  a  common  transversal  for  an  Ofn,  t) 
-set  if-the  eolleeUdn -is  a*  transversal'  for  eat;h  of“th«st'"t  ^latiW“iqaaMST'  SrmiTafIy 


two  common  transversals  are  said  to  be  parallel  if  they  have  no  cell  in  common. 
Example.  Tne  underlined  and  parcr.rnes  ized  cells  form  two  parallel  common 


tr.;r,:-vers  ns  to:  trie-  luiiowin*  C(4,2)  set. 


I  2  (3)  4 

J  (21  i  4  3 

]  3  (4)  1  2 

J  5  l  <1,1 


1  2(3)  4  ^ 

(4)3.  2  1  I 

2  (1)  4  3  ( 

3  4  i  (2), 


XJII.3.  Composing  Two  Latin  Squares  of  Order  n^  and  n^ 

A  very  natural  question  in  the  theory  of  latin  squares  is  the  following: 

Clive.’!  two  latin  squares  L,  and  1.  of  order  n,  and  n  (n,  >  n,)  respectively. 
In  hew  many  ways  can  one  compose  Lj  and  l  In  order  to  obtain  a  latin  square 
4  .A'i-'T  m,  wnerc  m  is  a  function  of  n.  and  n  only?  This  question 

5  1  e 

•,u  partially  answered  as  follows,  first,  it  is  well-known  that  the  Kroneeker 
;  ro'iin  t  X1  is  a  latm  square  of  order  m  »  n^n  irrespective  of  the 

comb  mate  rial  structure  of  and  !.,  .  Secondly,  we  show  that  If  Lj  has  a 
.  <  rt.un  ,  ..mc-inatonal  structup-,  then  one  can  construct  a  latin  square  L  of 


a: 


order  n  =  .  Naturally  enough  we  call  this  procedure  a  "method  of  sum 

composition 

Even  though  our  method  of  sum  composition  does  not  work,  for  all  pairs 
of  latin  squares,  it  has  an  immediate  application  in  the  construction  of  ortho¬ 
gonal  latin  squares  including  those  of  order  4t  +  2,  t  >  2  .  We  emphasize  that 
the  cornhihaibrial  structure  cf  orthogonal  latin  squares  constructed  by  the  method 
of  sum  composition  is  completely  different  from  those  of  known  orthogonal  latin 
squares  in  the  literature.  Therefore,  it  is  worthwhile  to  study  these  squares 
for  the  purpose  of  constructing  new  tinite  projective  planes. 

We  shall  now  describe  the  method  of  “sum  composition".  Let  and 

be  two  latin  squares  of  order  and  n2,  ,  on  two  non-intersecting 

sets  if.  *  {a.,  a, . a  }  and  1  .  =  {b.,b,,...,b  }  respectively.  If  L 

i  i  c  ci  c  i 

has  n2  parallel  transversals  then  we  can  compose  Lj  with  to  obtain  a 
latin  square  L  of  order  n  =  +  n2  ,  Note  that  for  any  pair  (n^,  n2>,  there 

exists  Lj  and  L2  with  the  above  requirement,  except  for  (2,1),  (2,2),  (6,5) 
and  (6,6)  . 

To  produce  L  put  and  L2  in  the  upper  left  and  lower  right  corner 
respectively.  Gall  the  resulting  square  C^,  which  looks  as  follows; 


Name  the  n2  transversals  of  Lj  in  any  manner  from  1  to  ,  Now  till  the 
cell  (i,  nj  +  k),  k  =  1, 2, , , ,  ,n2,  with  that  element  of  transversal  k  which 
appears  in  row  1,  i  =  1,  ,  Pill  also  the  cell  (n^  *  k  ,  j).  k  -  1,  2, ....  n 

tf  '  4 


with  that  element  of  transversal  k  which  appears  In  column  j,  j  =  1,2,.. .  ,n^ 
Call  the  leijiung  square  .  Now  every  entry  ot  is  occupied  with  an 
element  either  from  or  but  is  obviously  not  a  latln  square  on 
“1  ^2  •  However,  ^  we  replace  each  of  the  entries  of  transversal  k 

with  b^,  it  is  easily  verified  that  the  resulting  square  which  we  call  L  is  a 


“  la  tin“  square  o  f 'order  U  - - - ~ - - - - - 

The  procedure  described  for  filling  the  first  n^  entries  of  the  row  (column) 
*  k  with  the  corresponding  entries  of  transversal  k  is,  naturally  enough, 
eulicu  the  ut^jectiur.  ot  transversal  »;  on  the  first  n^  entries  of  row  (column) 
n^  *  k  . 

We  shall  now  elucidate  the  above  procedure  via  an  example.  Let  = 
{l.d.  3. 4.  S  },  L  ^  =  (6,7,8)  , 

V« 

/// 4  5 


/  1/2  3  •»’ 

ii  * 

i-i »  fir  i  i  }  Ar 

l  4  5  ‘A/l 


and  L2  = 


6  7  8 

7  8  6 

8  6  7 


Note  that  the  cells  on  the  same  curve  in  form  a  transversal. 


(7 

i 

3 

4 

5 

5 

1 

2 

3 

4 

4 

3 

1 

i 

l 

j 

3 

4 

5 

\ 

L 

) 

3 

4 

5 

1 

and  C. 


6  7  H 

7  8  6 


12  3  4  5 

1  2  3 

5  12  3  4 

4  5  1 

4  5  12  3 

2  3  4 

3  4  5  1  2 

5  1  2 

2  3  4  5  1 

3  4  5 

13  5  2  4 

6  7  8 

5  2  4  1  3 

7  8  6 

4  13  5  2 

8  6  7 

H  tl 


And  finally 


L  = 


which  Is  a  latin  square  of  order  8  on  2^  U  *  {1,  2,. . ,  ,8} . 

Remark.  Note  that  it  is  by  no  means  required  that  the  projection  of  transversals 
or.  the  rows  and  columns  should  have  the  same  ordering.  Indeed,  for  the  fixed 
set  of  orderea  n  transversals,  we  have  n  choices  of  projections  on  columns 

fa  fa 

and  n2!  choices  of  projections  on  the  rows.  Hence  we  can  generate  at  least 
(n^l  )2  different  latin  squares  of  order  n  =  ♦  n^,  composing  Lj  and  L 2  . 

XIII.  4.  Construction  of  Oin,2)  Sets  bv  Method  of  Sum  Composition.  In  order 
to  construct  an  0(n,2)  set  for  n  =  n^  ♦  n^,  we  require  that  rij  >  2n^  and 
there  should  exist  an  0(n2,2)  set,  and  an  0(^,2)  set  with  2n 2  parallel 
transversals.  It  is  easy  to  show  that  any  n  >  10  can  be  decomposed  in  at  least 
one  way  into  4  n^  which  fulfill  the  above  requirements.  We  now  present  two 
theorems  which  state  that  for  certain  n  one  can  construct  an  0(n,2)  set  by  the 
method  of  sum  composition. 

Theorem  XIII.  4. 1.  Let  =  p"  >  7  for  any  odd  prime  p  and  positive  integer 
a,  excluding  n1  =  13,  Then  there  exists  an  0(n,2)  set  which  can  be  constructed 
by  composition  of  two  0(^,2)  and  0(n2,2)  sets  for  n^  *  (n^D/2  _and 

n  "  nl  4  n2  * 


6 

7 

8 

4 

5 

1 

2 

3 

7 

8 

2 

3 

h 

4 

5 

1 

8 

5 

1 

b 

7 

2 

3 

4 

i 

4 

b 

7 

8 

t> 

1 

i 

2 

b 

7 

8 

l 

3 

4 

5 

1 

3 

5 

2 

4 

6 

7 

8 

5 

2 

4 

1 

3 

7 

8 

6 

4 

_  i 

3 

5 

2 

8 

6 

7 

We  shall  first  give  the  method  of  construction  and  then  a  proof  that  the 
constructed  set  is  an  CHn,2)  set. 

Construction.  Let  B(r)  be  the  r.^  x  square  with  element  ro  +  in  Its  (i,j) 
cell,  a.,  ,  0  *  r  in  GF(nj).  i,j  =  1,2,. . . , r^  .  Then  it  is  easy  to  see  that 

{  B(l)y  8(x),  Bty)},  y  *x  \  x  #  1,  is  an  0(nj,3)  set.  Consider  the  n}  ceils  in  B(l) 

-----  .  .  .  ‘  "V* 

with  *  k  a  fixed  element  in  GFfn^)  .  Then  the  corresponding  cells  in 

B(x  i  and  B,'yi  form  a  common  transversal  for  the  set  {B(XI,  B(y ) }  .  Name  this 
common  transversal  by  k  .  It  is  then  obvious  that  two  common  transversals  k^ 
and  k,,  kj  *  k^  are  parallel  ar.d  ^ence  {B(x>,  B(y)}  has  common  parallel 
transversals.  Now  let  {A^,A^}  beany  Oin^,2j  set,  which  always  exists,  on 
a  set  s.'  non-intersecting  with  GFin^j  .  Tor  any  k  in  GF(n^)  we  can  find 
irij-lt'2  pairs  of  distinct  elements  belonging  to  GFin^J  such  tfhat  the  sum  of 
the  two  elements  of  each  pair  is  equal  to  k  .  Let  {S}  and  {T}  denote  the 
collection  of  the  first  and  the  second  elements  of  these  (n^-l)/2  pairs  respec¬ 
tively.  Note  that  for  a  fixed  k  the  s--t  { S >  can  be  constructed  In  (n^-i)(n^  —  3> 
...  1  aictmct  ways.  Now  fix  \  and  let  L^  denote  any  of  the  (n^l  )  latin  ' 
squares  that  can  be  generated  oy  the  sum  composition  of  L(x)  and  using 
transversals  determined  by  the  elements  of  t S >  .  Let  L2  be  the  latin  square 
derive-,:  fr  .m  the  composition  of  l<  yi  and  A^  using  the  transversals  de- 
t»r.nr  ■  :  :  >•  me  elements  of  (T)  m  i  the  following  projection  rulei  Project  trans¬ 
versals  t  ,  i  =  1,2 . n^  on  the  row  (column)  which  upon  superposition  of 

L,  on  l.j  this  row  (column)  should  coincide  with  the  row  (column)  stemmed 
from  the  transversal  k-t  .  Shortly  we  shall  prove  that  {L  ,L  }  forms  on  0(n,2) 


The  preceding  arguments  shows  that  {L^.L^}  can  be  constructed  non- 
isomorphicaily  in  at  least  )*“[  n  (n^~ l)(n^— 3).  . .  1J  ways.  For  instance 

in  the  case  of  =  7,  there  is  at  least  1209b  non-isomorphic  pairs  of  ortho¬ 
gonal  latin  squares  of  order  10  .  Therefore,  Euler  has  been  wrong  in  his  con- 
lecture  by  a  very  wide  margin.  . 

Mote  that  we  can  construct  infinitely  many  ba'irs  of  orthogonal  latin  squares 
of  order  4t  +  2  by  the  method  of  theorem  XIII.  4. 1.  For  p?7  mod  8  and  a  odd 
p°  =  (Bt  ■*  51/3)  .  Hence  n^  +  n^  =  4t  -*  2  . 

Proof:  The  constructional  procedure  clearly  reveals  that: 

A.  and  are  latin  squares  of  order  n  on  GF(n^)  U  si  . 

B.  Upon  superposition  of  on  the  following  are  true: 

bj.  Every  element  of  £2  appears  with  every  other  element  of  £2  . 

b^.  Every  element  of  £2  appears  with  every  element  of  GF(n^)  . 

\ 

bj.  Every  element  of  C^F^i  appears  with  every  element  of  £2  . 

Therefore,  all  we  have  to  provo  is  that  every  element  of  GF(n^)  appears' with 
every  other  element  of  GUn^)  .  To  prove  tins  recall  that  Bix)  is  orthogonal  to 
Biy).  However,  since  we  removed  the  n,  transversals  from  B(x)  determined 
by  the  n^  elements  of  {S}  and  n^  transversals  from  B(y)  determined  by  the 
n2  elements  of  {1}  therefore  the  following  2n^n^  pairs  have  been  lost. 

(xuj  *  a yo.  +  )  with  a.  *  a)  =  'i  for  any  \  «  GF‘ (n^ ) ,  \  \  . 

We  claim  that  the  given  projection  rules  guarantee  the  capture  of  these  lost  pairs 
by  the  Zn^hj  bordered  cells.  To  show  this  note  that  the  superposition  of  the 


^4  *  t 

y  '  j  *. 


projected  transversal  s  from  B(x)  on  the  projected  transversal  t  *  X  -s  from 
B(y )  will  capture  the  pairs. 

(xa.  +  oy  yo^  +  e  ■)  with  a.  ■+  -  k  =  [y(K-s)  +  s]/(l+y) 

\it  these  transversals  have  been  projected  on  row  border  and  n^  ‘pairs 

if  these  transversals  have  been  projected  on  column  border.  Now  because 
k  i  k1  =  k  and  if  s  *  then  kj  *  k^  and  kj  ¥  k^  hence  the  2n2nj  pairs 
w’ht.'h  nave  been  resulted  from  the  projection  of  transversals  determined  by  {S } 
and  { T }  will  jointly  capture  the  ^n^n|  l°st  Pa*rs  an<^  fhus  a  proof. 

We  shall  now  clarify  the  above  constructional  procedure  by  an  example. 

■  1 

Example.  Let  ^  =  7,  GF(7)  {0.1,2 . 6}.  Then  for  x  =  2,  y  «  x  =4 

we  have 


{Bill,  Bi2j.  B;4)}  « 

0  12  3  4  5  6 

12  3  4  3  6  0 

2  3  4  6  h  0  I 

3  4  6  (I  0  1  2 

4  6  0  1  2  3 

6  6  f)  1  2  3  4 

0  0:2  3  4  5 


0  12  3  4  5  6 

2  3  4  5  6  0  1 

4  5  6  0  12  3 
6  0  12  3  4  5 

12  3  4  5  6  0 

3  4  5  6  0  12 

6  6  0  1  2  3  4 


0  12  3  4  5  6 

4  5  6  0  1  2  3 

12  3  4  5  6  0 

5  6  0  1  2  3  4 

2  3  4  5  6  0  1 

6  0  12  3  4  5 

3  4  5  6  0  1  2 


Ter  :i,  =  in  -l>/2  -  3  let  »2  -  {7,8,9}  and 
7  h  9  7  8  >1 

iA  .A,;  -  8  7,  9  7  8.  Finally  tor  \  =  0,  { S }  =  {1,2,3}  and 

1  1  4  7  8  8  9  7 


. Tj  ::  {8,5.4}  we  have  {L^,  L,}  = 
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1 

4 

3 

2 

3 

4 

5 

? 
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the  reader  can  easily  verify  that  {L  ,  L  }  is  an  0(10,2)  set. 

1  ^ 

Remarks. 

1)  The  method  of  theorem  XIII.  4.1  fails  for  =  13  only  because  there  is  no 
0(6,2)  set.  Otherwise,  there  will  be  no  orthogonality  contradiction  on  the  other 
parts  of  Lj  and  with  their  6x6  lower  right  square  missing. 

2)  In  the  case  of  n  =  7,  if  we  let  IS}  =  {0,1,3}  and  (T)  =  {2,4,5}  then  the 

*  '..-  i 

requirement  y  =  x  *  is  net  necessary.  Ilovyever  then  we  do  not  have  a  unified 

i 

projection  rule  for  the  formation  of  gs  was  provided  for  the  case  y  =  x  1  by 
theorem  XIII.  4,  l.  To  give  the  complete  list  of  solutions  let  (a^a^.a^  and 
(brb2,b3)  be  any  two  permutations  of  the  set  {8,9,10}  .  If  we  project  trans¬ 
versals  (0,1,3)  on  the  rows  (a,,a,,a  )  and  columns  (b,,b,,bj  in  the  forma- 

l  &  5  1  j 

tion  of  1^,  then  the  following  table  indicates  what  permutation  of  transversals 
{2,4,5}  should  be  projected  on  the  rbws  (a  ,a,,a3)  and  columns  (b  ^ .  b  ^ ) 
in  the  formation  of  .  Obviously  these  permutations  will  be  a  function  of  the 
pair  (x,y)  . 


1  4  *  #  ' 

4  1 


! 

(This  table  is  by  no  means  exhaustive.  )  « 

’  -1 

The  reader  may  note  that  whenever  y  =  x  in  the  above  table  the  given  1 

i 

solutionis )  are  different  from  the  one  provided  by  the  method  of  theorem  XIII.  4.1.  j 

Thus  we  can  conclude  that  any  pair  of  orthogonal  latin  square*  of  order  7 
based  on  the  GTI?  i  can  be  composed  with  a  pair  of  orthogonal  latin  squares  of 

/  •  s  1 

</  •  ‘  JL 

I 


/ 


order  3  and  make  a  pair  of  orthogonal  latin  squares  of  order  10  .  In  addition, 

f 

since  we  have  six  choices  for  (a^a^a^)  and  (b^b^fc^)  hence  from  every 
line  in  the  above  table  we  can  produce  36  non- isomorphic  0(10,2)  sets  or 
16  x  36  *  576  sets  for  the  entire  table.  Since  all  these  pairs  are  non-isomorphic 


with  all, previous  pairs*  produced  by  theorem  XII|  .4,1,.  thus  by  the  method  of  sum 


composition  one  can  at  least  produce  12, 672  non-isomOrphic  0(10,2)  sets. 

We  believe  that  for  other  values  of  n^  there  are  sets  of  {S}  and  {T} 
together  with  proper  projections  which  makes  the  restriction  y  =  x  *  unnecessary. 
Theorem  XIII.  4.  2.  Let  n^  =  2°  >  8  for  anv  positive  integer  a  .  Then  there 
exists  an  0(n,2)  set  which  can  be  constructed  by  composition  of  two  0(^,2) 
and  0(n2,2)  sets  for  n2  =  nj/2  and  n  »'  4  n2  . 

We  shall  here  give  only  the  method  of  constmction.  A  similar  argument 
as  in  theorem  XIII.  4.1  will  show  that  the  constructed  sot  is  an  0(n,2)  set. 
Construction.  In  a  similar  fashion  as  in  theorem  XIII.  4. 1  construct  the  set 
C B( i: ,  B(x),  B(y >>  over  GF(2a)  .  Let  also  {A^A.,}  be  any  0(n2,2)  set,  which 
always  exists,  on  a  set  12  non-intersecting  with  GF(2a)  .  For  any  X.  *  0  in 
GF(2a)  we  can  find  n^/2  pairs  of  distinct  elements  belonging  to  GF(2°)  such 
that  the  sum  of  the  two  elements  of  each  pair  is  equal  to  X  .  Let  {S}  and  {T } 
denote  the  collection  of  the  first  and  the  second  elements  of  these  n^/2  pairs 
respectively.  Note  that  for  a  fixed  x  the  set  { S }  can  be  constructed  in 
n^(n -2i(n^-4). . .  1  distinct  ways.  Now  form  Lj  from  the  sum  composition  of  B(x) 
and  Aa  and  L2  from  the  sum  composition  of  B(y)  and  A2  using  the  same  pro¬ 
jection  rule  as  given  in  theorem  XIII.  4. 1.  Now  {L^.Lg}  is  an  0(n,2)  set. 


I.  •  t 

*969*  1 


—  I?iple.  Let  n  =  8.  GF(8)  =  {0,1,2,. ..  ,7}  with  the  following  addition 
and  multiplication  (X  )  tables: 
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Then  for  x  =  2 ,  y  =  x"1  =  7  we  have 

{ B(  1 ) ,  B(2),  B(7 )}  = 

01234567  012 

1  0  6  4  3  7  2  5  260 

26075413  347 

3. .4  701652  435 

43510276  5  74 

57462031  6  21 

62157  3  04  753 

7  5  3  2  6  1  4  0  1  ‘0  6 
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For  =  rij/2 


4  let  Si  =  (A,  B,  C,  D}  and 


A  B  C  D 
B  A  D  C 
G  D  A  B 
Lj  C  R  A 


A  B  C  D 
D  C  B  A 
B  A  D  C 
C  D  A  B 


A  B  2  C  D  5  6  7 

0  13  4 

BAOOC413 

6  2  5  7 

3  4  A  0  1  D  B  C 

7  5  2  6 

CD5AB27  6 

10  4  3 

D  C  4  B  A  0  3  l 

2  6  7  5 

62D57ACB 

3  4  0  1 

75B26CAD 

4  3  10 

10  C  4  3  B  D  A 

5  7  6  2 

FTTTTTTT 

A  B  C  D 

21360754 

B  A  D  C 

4  7  6  3  5  1  0  2 

C  D  A  B 

5  3  1  7  4  6  2  0 

0  C  B  A 

which  is  an  Oil2,2)  set. 
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A  B  5 
C  D  4 
B  A  1 
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6  3  0 
5  0  3 
l  7  2 


.21  we  have  {L  .  L 

1  4» 


4  A  C 
6  B  D 
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5  C  A 
D  B 

2  7 
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5  7 
0  1 

3  4 
1  0 

C|  7  5 

4  3 

6  2 

WIT 


l  o 
3  7 
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17  4  6 
5  0  6  4 


rr 

D  C  B  A 
B  A  D  C 
C  0  A  B 


*  4  • 


^t^v-uccion.  The  necessary  requirements  for  the  construction  uf 


an 


n  3  nj  4  n2i  t  <  n2,  by  the  method  of  sum  composition  are:  The  existence  of  an 
0(n^,t)  set,  n^>tn2,  with  at  least  tn^  common  parallel  transversals,  and  an 

0(n  ,t)  set.  These  conditions  are  obviously  satisfied  whenever  n,  and  n„ 

,•  .  *• — -  -  ...  . .  ,  _  ,  -  _.  ...  .  1  _  Z _ 

are  prime  powers.  - — :  - - •  : '  — — — — 


While  for  some  values  of  n  there  exists  only  a  unique  decomposition  ful¬ 
filling  the  above  requirements,  for  infinitely  many  other  values  of  n  there  are 


abundant  such  decompositions. 

It  seems  that  if  there  exists  an  0(n.,2)  set  and  if  n  *  n.  +  n,,  n.  >  2n, 
then  one  can  construct  an  0(n,2)  sot  by  the  method  of  sum  composition  if  n^ 
is  a  prime  power.  To  support  this  observation  and  shed  some  more  light  on  the 
method  of  sum  composition  we  present  in  subsequent  pages  some  highlights  of  the 
results  which  we  hope  to  complete  and  submit  for  publication  shortly. 


In  the  following  for  each  given  decomposition  of  n  we  exhibit  an  0(n,2) 


set  which  has  been  derived  by  the  method  of  sum  composition.  We  shall  represent 


the  pairs  in. a  form  that  the  curious  reader  can  easily  reconstruct  the  original  sets. 
Hereafter  the  notation  j  means  that  is  orthogonal  to  . 

1)  12  *  9  *  3 


ABC436789 

123  123456ABC 

8  9  7 

B  C  A  1  2  3  4  5  6 

9  7  B  9  7  6  3  1  2  1  CA 

6  4  5 
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564  564897CAB 

1  2  3 

231364ABC 

897  645ABC312 

7  8  9 

8  9  7  2  3  1  »  C  A 

456  231BCA897 

5  6  4 

5  6  4  1  9  7  CAB 

312  j.  789CAB4S6 

3  1  2 

3  1  2  A  1  C  1  7  I 

645  ABC231564 

9  7  6 

9  7  8  »  C  A  6  4  5 

231  1  C  A  7  1  9  1  2  3 

4  3  6 

6  4  5  CAB)  12 

i  s . 9  9 A  !  H  3  ?  7  S 

2  3  1 

TTTTTTTTT 

tie  fTTTTTiTT 

A  1  C 

726341391 

B  C  A  337166249 

CAB 

413915267 

CAB  616924733 

B  C  A 

✓  ' )  . 


2)  14  =  ll  4  3,  the  only  decomposition  which  fulfills  the  necessary  require-* 


ments. 
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01234567ABC9  10  8 

89  10  0123ABC7564 

3  6  7  8  1  10  A  1  0  9  4  1  H 

2  3  4  3  6  A  B  C  10  0  1  8 
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ABC12345678  10  09 

B  C  8  910  01  234A67  3 

C  4  3  6  7  B9  10  0AB231 

6  10  3704I1592ABC 

3704813926  10  CAB 


J)  IS  -  12  *  3,  15  -  11  *  4  are  the  only  Jecoii:p>sltlons  which  fulfill  the  neces¬ 
sary  requirements,  However,  wo  consider  here  the  latter  decomposition  since 
wo  can  utilise  the  properties  of  Galois  field  GFfil), 
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are  the  only  decompositions  which  fulfill  the  .necessary  requirements 
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do  not  know  whether  there  exists  either  an'  0(18,2)  set  with  8  common 
parallel  transversals  or  an  0(15,2)  set  with  14  common  parallel  trans¬ 
versals.  Therefore  the  only  decomposition  of  22  which  fulfill  the  necessary 
requirements  are  22  =  l<?  +  t  and  22  s  17  *  A 


a:  22  *  19  +  3, 

A  8  C  3  4  5  6 

B  C  5  6  7  8  9 

C  7  8  9  10  11  12 

9  10  11  12  13  14  15 

12  13  14  *5  1*17  I  t 

15  16  17  1#  0  12 

18  0  1  2  3  4  5 

2  3  4  5  6  7  8 

5678  9  10  11 
8  9  10  11  12  13  14 

11  12  13  14  IS  16  17 

14  15  16  17  18  0  l 

17  18  0  1  2  3  4 

1  2  3  4  5  6  A 

4  5  6  7  8  A  B 

7  8  9  10  A  B  C 

10  11  12  A  B  C  16 

13  14  A  B  C  18  0 

16  A  B  C  1  2  3 


10  11  12  A  B  C  16 
13  14  A  B  C  18  0 
16  A  B  C  1  2  3 

Trnnnrn 

3  1  18  16  14  12  10 
8  4  2  0  17  IS  13 


7  8  9 
10  11  12 

13  14  15 

16  17  18 
0  t  2 

3  4  5 
6  7  8 
9  10  11 

12  13  14 
15  16  17 
18  0  A 
2  A  B 
ABC 
B  C  10 
C  12  13 

14  15  16 

17  18  0 
l  2  3 

4  S  6 


10  11 
13  14 
16  17 
0  1 
3  4 
6  7 
9  10 
12  13 
15  A 


11  12 
14  15 
17  18 


18  16 
2  0 


12  13  14 

15  16  17 
18  6  1 

2  3  4 

5  6  7 

8  9  4 
11  A  B 

ABC 
B  C  0 
C  2  3 
4  5  6 
7  B  9 
10  11  12 

13  14  15 

16  17  18 
0  12 

3  4  5 

6  7  8 

9  10  11 

14  12  10 

17  15  13 
1  18  16 


15  16 
18  0 


17  18 
1  2 


10  11 
13  14 
16  17 
0  1 


9  10 
12  13 


17  18 
1  A 
A  B 
B  C 

e  ii 

13  u 
16  17 
0  1 


9  10 
12  13 
15  16 
18  0 


11  12 
14  15 


14  12 


0  12 

2  3  4 

4  5  6 

6  7  8 

8  9  10 
10  11  12 

12  13  14 

14  15  16 

16  17  16 
18  0  1 

12  3 

3  4  5 

5  6  7 

7  8  9 

9  10  11 
11  12  13 

13  14  15 

15  16  17 

17  18  0 

“A" . i""  fl 

B  C  A 
CAB 


0  1 

13  14 
7  8 
1  2 

14  15 


2  3 
15  16 


15  16 
9  10 

3  4 

16  17 
10  11 

4  A 


16  17 
10  11 

4  5 

17  18 
11  12 

5  6 


12  13 
6  7 
11  16 
5  12 


4  5 

17  18 
11  12 

5  6 

18  0 
12  13 


13  14 
7  8 
1  2 

14  15 


2  3 
15  16 


16  17 
10  11 
1  8 
14  2 
7  14 


6  7  8 
0  1  2 

13  14  IS 

7  8  9 
1  2  3 

14  15  A 

8  A  8 
ABC 
B  C  17 
C  10  11 

3  4  5 

16  17  18 
10  11  12 

4  5  6 

17  18  0 

11  12  13 

5  6  7 

18  0  1 

12  13  14 

15  3  10 

9  16  4 


16  17 


11  12 
5  A 
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12  13 


13  14 
7  8 
1  2 

14  15 


13  14 
7  8 
l  2 

14  15 


15  16 
17  5 
11  18 
4  11 


15  16 
9  10 

3  4 

16  17 
10  11 

4  5 

17  18 
12  0 

6  13 


14  15 
8  9 
2  3 

15  16 


16  17 
10  11 

4  5 
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5  6 
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12  13 
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14  15 


15  16 
9  A 
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10  17 
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22  -  17  +  3, 


A  B 
B  C 

C  D 
D  E 
E  9 
10  U 
12  13 
14  15 


9  10 
11  12 
13  14 


CDS 
DEB 
E  7  8 

8  9  10 
10  11  12 

12  13  14 
14  15  16 
16  0  1 

12  3 
3  4  5 
5  6  7 
7  8  9 

9  10  11 
11  12  A 

13  A  B 
ABC 


5  6 
7  8 
9  10 
11  12 
13  14 
15  16 
0  1 


10  A 
A  B 


7  8  9 
9  10  11 
11  12  13 
13  14  15 
15  16  0 
0  12 

2  8  4 
4  5  6 
6  7  A 
SAB 
ABC 
BCD 
C  D  E 
DEI 
E  2  3 

3  4  5 


10  11 
12  13 
14  15 
16  0 


12  13  14 

14  IS  16 
16  0  1 

1  2  A 

3  A  B 
ABC 
BCD 

e  D  E 
D  B  13 
E  14  15 

15  16  0 
0  12 

2  3  4 

4  5  6 

6  7  8 
8  9  10 


0  16  15  14  13  12  11  10  9  8  7  6  5  4 

2  1  0  16  15  14  13  12  11  10  9  8  7  6 

4  3  2  1  0  16  15  14  13  12  11  10  9  8 

6  5  4  3  2  1  0  16  15  14  13  12  11  10 


0  1 
9  10 
1  2 
10  11 

2  3 

11  12. 

3  4 
12  13 

4  A 
A  B 
B  C 
C  D 
D  E 
E  16 
7  8 

16  0 


2  3 
11  12 

3  4 

12  13 

4  5 

13  14 

5  A 


4  5  6 

13  14  15 

5  6  7 

14  15  A 

6  A  B 
ABC 
BCD 
C  D  E 
D  E  10 
E  1  2 
9  10  11 

1  2  3 
10  11  12 

2  3  4 
11  12  13 

3  4  5 


7  8 
16  A 
A  B 
B  C 
C  D 
D  E 
E  11 

2  3 
11  12 

3  4 

12  13 

4  5 

13  14 

5  6 

14  15 

6  7 


A  B 
B  C 
C  D 
D  E 
E  12 

3  4 

12  13 

4  5 

13  14 

5  6 

14  15 

6  7 

15  16 

7  8 

16  0 

8  9 


C  D 
D  B 
E  13 

4  5 

13  14 

5  6 

14  15 

6  7 

15  M 

7  8 

16  0 

8  8 
0  1 
9  10 
1  A 
A  B 


15  16 
0  A 


E  11  6 
12  13  7 
14  15  8 
16  0  9 
1  2  10 
3  4  11 
5  6  12 
7  8  13 
9  10  14 
11  12  15 


B  9  10  11 
9  16  11  12 

10  11  12  13 

11  12  13  14 

12  13  14  15 

13  14  15  16 

14  15  16  0 

15  16  0  1 

16  0  1  2 


rciim 
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13  14 
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16  12 
7  3 
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14  10 


3  12  4  13ABCDE 
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4  13  5  14DBABC 
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XIV.  Computer  Construction  of  CX 10,  t)  Sots 
in  about  fifteen  years  the  effectiveness  of  computers  in  searching  for 
orthogonal  sets  of  latin  squares  of  order  ten  has  increased  strikingly.  Still 
th?  problem  is  so  large  that  there  seems  to  be  little  reason  for  optimism  that  the 
ofder.ten  problem  can  be  completed  by  computers.  .More  precisely,  -^ {as  most  , - 
conversant  with  the  problem  consider  quite  plausible)  no  orthogonal  triple  of 
orthogonal  latin  squares  of  order  ten  exists,  then  the  number  of  cases  to  con¬ 
sider  seems  too  large  for  an  exhaustive  proof  by  computer  to  be  achievable.  The 
number  of  latin  squares  of  order  ten  is  astronomical. 

About  19 Paige  am;  Tompkins  (195s]  programmed  SWAC  to  search  for 
squares  orthogonal  to  a  fixed  latin  square  of  order  ten.  A  few  hours  of  running 
produced  no  orthogonal  square,  and  was  regarded  as  a  bit  of  experimental  evi¬ 
dence  lor  the  truth  cf  Euler's  conjecture.  Calculations  based  on  the  progress 
mu'.:  ■  in  the  search  led  tc  the  extrapolation  that  over  fifty  million  years  of  computer 
time  would  be  required  to' search  for  all  squares  orthogonal  to  a  latin  square  of 
order  ten  put  into  SWAC  intially.  (At  about  the  same  time  a  similar  program  was. 
written  and  similar  results  obtai  -i:.:.  with  MANIAC  at  Los  Alamos;  this  attempt 
not  been  reported  in  print.) 

In  19  S9 ,  after  Euler's  conjecture  had  been  disproved  for  all  orders 
4!  .  ’  ■  b,  Parker  programmed  UN  I  VAC  1206  to  search  foi  squares  orthogonal 
u  a  latin  square  of  order  ten.  The  running  time  was  sharply  less  than  for  SWAC 
or  MANIAC,  about  thirty  minutes  for  the  majority  of  latln  squares.  This  was 
accomplished  by  generating  and  storing  all  transversals  of  the  input  latin  square, 


then  searching  for  all  ways  to  form  latin  squares  from  the  list  of  transversals. 

(A  transversal,  or  directrix,  is  a  set  of  cells  of  a  latin  square,  one  in  each  row, 
one  in  each  column,  and  one  containing  each  digit. )  The  striking  gain  in  spued 
over  the  earlier  efforts  occurred  largely  because  the  number  of  transversals  of 
a  typicai  rstlh  square  oT  order  Tten  ie  roughly  850,  much  less  than  40V  ?  end-* 
of  course,  the  search  was  several  levels  deep.  (SWA C  and  MANIAC  were  pro¬ 
grammed  to  build  up  starts  of  latin  squares  toward  orthogonal  mates  by  filling  in 
colls  to  form  rows.  ) 

There  were  two  main  outcomes  from  considerable  running  of  Parker's  1206  ./ 

•  / 

program:  1)  Orthogonal  triples  of  order  ten  latin  squares  are  not  numerous;  more  / 

/ 

precisely,  only  a  small  fraction,  if  any,  order  ten  squares  extend  to  ttiples.  / 

Some  400  latin  squares  were  run.  Some  were  random,  some  were  computer  output 
fed  back  as  input  and  hence  known  to  have  ah  orthogonal  mate,  and  some  were 
considered  interesting  candidates  for  intuitive  reasons  by  Parker  and  others. 

Not  cnce  did  an  exhaustive  search  for  orthogonal  mates  of  an  Input  latin  square 
include  a  pair  orthogonal  to  one  another.  Mild  evidence  may  be  claimed  sup¬ 
porting  the  opinion  that  no  order-ten  orthogonal  triple  exists.  2)  Of  a  computer¬ 
generated  sample  of  100  random  latin  squares  of  order  ten  (program  by  R.  T. 

Ostrowski),  62  have  orthogonal  mates.  Thus,  unlike  triples,  order  ten  ortho¬ 
gonal  pairs  are  quite  common.  Euler's  intuition  for  order  ten  was  not  only  wrong, 
but  In  this  sense  wrong  by  a  large  margin.  It  was  this  finding  which  tempted 
Parker  for  a  time  to  believe  that  repeated  runs  of  the  program  should  have  a  good 
chance  of  producing  at  least  a  triple,  but  many  failures  dimmed  this  optimism. 


f- 


!n  196?  John  W.  Brcwr.  programmed  13  M  7094  to  decide  whether  an  input 
latin  square  of  order  tan  can  be  extended  to  an  orthogonal  triple.  The  running 
time  was  one  half  minute.  Almost  needless  to  say,  transversals  again  were 
.generated.  Searching  for  patterns  of  transversals  toward  extension  to  a  triple 
produced  a  speed  gain  over  the  previous  program  for  orthogonal  pairs.  ifoWn 
endeavored  to  get  every  drop  of  speed  from  the  machine.  As  before,  hundreds 
of  input  order-ten  latin  squares  produced  no  orthogonal  triple. 


/ 
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On  the  Equivalence  of  O(n.t)  Sets  With  Other  Combinatorial  Systems 
XV.  o.  Summary 

In  this  section  we  have  densely  summarized  some  of  the  results  obtained 
by  author  and  at  least  fourteen  others  in  order  to  demonstrate  the  importance  of 

squares  *  We  ha  ve  shown  that  fdurteeh 
well-known  and  important  combinatorial  systems  with  certain  parameters  are 
actually  equivalent  to  a  set  of  mutually  orthogonal  latin  squares.  A  schematic 
representation  of  these  equivalences  has  been  demonstrated  in  four  wheels  which 
we  have  called  "Fundamental  Wheels  of  Combinatorial  Mathematics". 

XV.  1.  Introduction 

The  theory  of  mutually  orthogonal  latin  squares  owes  its  importance  to  the 
fact  that  many  well-known  combinatorial  systems  are  actually  equivalent  to  a 
set  of  mutually  orthogonal  latin  squares;  viz.,  finite  projective  plane,  finite 
Euclidean  plane,  net,  BIB,.  PBIB,  orthogonal  arrays,  a  set  of  mutually  orthogonal 
matrices,  error  correcting  codes,  strongly  regular  graphs,  complete  graphs,  a 
balanced  set  of  f-restrictional  lattice  designs,  difference  sets,  Hadamard 
matrices,  and  an  arrangement  of  non  attacking  rooks  on  hyperdimensional  chess 
board.  These  combinatorial  systems  are  unquestionably  potent  and  effective  in 
all  branches  of  combinatorial  mathematics,  and  in  particular,  in  the  construction 
of  experimental  designs.  Therefore,  a  statement  that  the  theory  of  mutually 
orthogonal  latin  squares  is  perhaps  the  most  important  theory  in  the  field  of 
experiment  designs  is  not  in  the  least  exaggerated  as  far  as  this  author  is  con¬ 
cerned. 
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Our  purpose  in  this  section  is  to  demonstrate  the  relation  of  a  set  of 
mutually  orthogonal  latin  squares  with  the  above  mentioned  combinatorial  systems. 
We  shall  present  the  essence  of  the  known  results  available  only  In  scattered 
literature  in  one  theorem  which  we  consider  to  be  a  “fundamental  theorem  of 
combinatorial  mathematics".  Tor  the  definitions  of  these  combinatorial  systems  ; 
;^ajn|rthe.i^^;ifeiortii^roing:ih^rem:=sfie.theJl8t-6f.references.=§iveft-#Mfi©--^ 
end  of  this  paper. 

XV.  i.  Notation 

for  th‘-  sake  <  f  conciseness  wc  introduce  the  following  nutations; 

0)  CMn.t)  denotes  a  set  of  t  mutually  orthogonal  latin  squares  of  order 
n  . 

1)  MOMfn.t)  denotes  a  set  of  t  mutually  orthogonal  nXn  matrices. 

2 

d)  i.)A(n,t)  denotes  a  set  of  orthogonal  arrays  of  size  n  ,  depth  t,  n 

0 

levels,  and  strength  i  . 

«  Ji  Net(n,t)  denotes  a  net  of  order  n  and  degree  t,  '* 

■4)  '.Jodetn  ,  r,  t  :m  )  denotes  a  s-t  of  n  code  words  each  of  length  r  such 
that  any  two  code  words  are  at  least  at  Hamming  distance  >  't  on  an 
m-SHt  with  m  distinct  'dements.  We  remind  the  reader  that  such 
a  code  is  also  celled  it-li-orror  detecting  code  or  (t—l)/2— error  cor¬ 
recting  code  because  such  a  code  is  capable  of  detecting  up  to  t-1 
•■r..  r.s  an  i  ooirect  up  b,  ( t-1 1 /  i  errors  In  each  transmitted  code  word. 

Si  FBIBin ,  v ,  r ,  k ,  \  j  ,  \  ^  i  denotes  a  purtially  balanced  incomplete  block 

design  with  b  blocks  each  of  size  k,  v  treatments  with  r  replication 
>1  •■ach,  and  association  indices  \j  and  X^  . 


6)  SR-Graph  (A)  denotes  the  strongly  regular  graph  with  Incidence 
matrix  A  . 

7)  Non  #(n,t)  denotes  an  arrangement  of  n  mutually  non  attacking 
rooks  on  the  t-dimensianal  n  x  n  chess  board. 

.  8)  PG(2,s)  denotes  a  finite  projective  plaifte  of  order  s  (net  necessarily 

Desarguessian ). 

9)  d(2,s)  denotes  a  finite  Euclidean  plane  of  order  s  . 

10)  BIB(b, v,r,k,\ )  denotes  a  balanced  incomplete  block  design  with  b 
blocks  each  of  size  k,  v  treatments  with  r  replications  of  each,  and 
association  index  k  . 

Ill  K-Graph  (A)  denotes  the  complete  graph  with  incidence  matrix  A  . 

12)  DIF(v,k,k)  denotes  a  difference  set  with  parameters  v,  k,  and  k  . 

13)  BLRL(s)  .denotes  a  balanced  set  of  f-restrictional  lattice  design  for 

s  treatments.  Note  that  a  1-rostrictional  balanced  lattice  designs  for 
simply  a  BIB  design. 

14)  HAD(n)  denotes  a  symmetric  normalized  Hadamard  matrix  of  order  n  . 
Hereafter  we  also  adopt  the  following  two  notations: 

i)  A  «*■»*»  B  means  A  implies  B  and  B  implies  A  , 

11)  A  B  means  A  implies  B  .  Whether  or  not  B  implies  A  is 


undecided. 


2.  The  Result: 

Theorem 


(a)  for  anv  pair  of  positive  Integers  n  and  t  we  havei 
i)  0(n,t)  **£«  MOM(n,t+2) 

. d\  0(n,t.)  <«>.  0A{n,t+2>  .  -  - .  •  - - 

3)  0(n,t)  <s=s>  Net(n,t+2) 

2 

4)  0(n,t)  Code(n  ,t+2,  t+ljn) 

2 

*> l  Oln,t)  PBIBfn  ,n(t+2l,  n,t+2,0,l) 

6)  O(n.t)  <s=o  SR-Graph  (A)  where  A  la  the  Incidence  matrix  asjsociated 
with  PBIB  in  5).  ' 


7) 

Otn.t)  « 

<■»  Non  *(n  ,  n  )  . 

11 

t  s  r, — 1  then  also: 

H  1 

Gm,n-li 

«*>  PG(2,ni 

■ 

n  i 

CMn . n-1) 

( 2 ,  n  > 

iu ) 

Qtn ,  n-1) 

2  2 

<*=>  BIB( n  +n  +  l,  n  +n+l, 

n+1,  n+1,1) 

ii  1 

Gin ,  n-1) 

i  2 

«sbb>  Codein  +n+i,  n  +  n+i 

,  2n;2) 

12) 

L’in ,  n-l) 

e**’  K-Graph  (A)  wnere 

A  is  the  incidence 

matrix  associated  with  BIB  in  10) 

1  .*■  1 

O'n , n-i) 

<**>  DlFtr/+n+l,  n+l,l>  , 

i  - 1  if  *  p™  where  p  is  a  prime  and  m  is  a  positive  integer  then  also  the 

lh  wiii'j ! 

mm  m 

14 1  O.p  ,p  -i)  BLRLip  )  . 
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(d)  If  n  =  2r  and  t  *  r-2,  r  >  3  then  the  following  are  also  true; 

15)  0(2r,r-2)  *->HAD(4r2) 

16  J  0{2rfr -2)  BIB(4r2  - 1 ,  4r2-l,  2r2-l,  2r2-l,  r2-l) 

17)  0(2r,r-2)  — 1>  Oode(4r2~l,  4r2-l,  2r2;2) 

18)  0(2r,r-2)  w*o  Code(8r2  ,  4r2,  2r2;  2)^ 

"  l$f  - 

A  complete  schematic  representation  of  this  theorem  can  be  demonstrated 
ir.  fuur  wheels  which  will  be  called  "fundamental  wheels  of  combinatorial  mathe 
matics".  For  the  sake  of  compactness  we  shall  omit  the  associated  parameters 
with  each  system  in  these  wheels  except  for  0(n,t)  .  By  knowing  the  values 
of  n  and  t  in  the  given  0(n,t)  sets, then  the  reader  can  easily  find  the 
associated  parameters  with  other  systems  in  the  wheels  from  the  proper  part  of 


above  theorem. 


XVI.  Acknowledgements 


• mr 


xvti. 


SfeSElB: 


1.  ueckenbach,  L.  t.  (ea.j  [i9b4{,  Applied  combinatorial  Mathematics. 

John  Wiley  and  Sons,  Inc.,  New  York,  London,  Sydney. 

2.  Bose,  R.  C.  f  19  39  |  ,  On  the  application  of  the  properties  of  Galois  Fields 
to  the  problem  of  construction  of  Hyper-Graeco- Latin  squares,  Sarikhya  3  : 


323-338, 


:rr.y--.rncr~T-  «■  tvjj 


3.  Bose,  R.  C.  [1939],  On  the  construction  of  balanced  incomplete  block 
designs,  Annals  of  Eugenics  9  :  343-399 . 

4.  Bose,  R.  C.  [1950),  A  note  on  orthogonal  arrays  (abstract).  Annals  of 
Mathematical  Statistics  21:  304-305. 

5.  Bose,  R.  C.  [19  63  ],  Strongly  regular  graphs,  partial  geometries  and 
partially  balanced  designs,  Pacific  Journal  of  Mathematics  13:  389-419. 

6.  Bose,  R.  C.  Chakravarti,  1.  M.,  and  Knuth,  D.  E.  [i960),  On  methods 
of  constructing  sets  of  mutually  orthogonal  Latin  squares  using  a  computer, 
1.  Technometrics  2:  507-516. 

i 

7.  Bose,  R.  C.  and  Clatworthy,  W.  H.  [1955/),  Some  classes  of  partially 

/ 

balanced  designs,  Annals  of  Mathematical  Statistics  26:  212-232. 

8.  Bose,  R.  C.  and  Shrlkhande,  S.  S.  [  19  59  )  ,  On  the  falsity  of  Euler's  con- 
lecture  about  the  non-existence  of  two  orthogonal  Latin  squares  of  order 
4t.  4  2,  Proceedings  of  the  National  Academy  of  Sciences,  U.S.A.  45: 
734-737. 


yn-i'  •y  v  *- 

V  )  I 


Bose,  R.  C.  and  Shrikhande,  S.  S.  [i960)  ,  On  the  construction  of  pairwise 
orthogonal  Latin  squares  and  the  falsity  of  a  conjecture  of  Euler,  Transac¬ 
tions  of  the  American  Mathematical  Society  9  5:  191-209. 

Bose,  R.  C.,  Shrikhande,  S.  S.  and  Parker,  E.  T.  [1960],  Further  results 
on  the  construction  of  mutually  orthogonal  Latin  squares  and  the  falsity  of 


r.-vyjf.  -* 


^Eulecf s :  taonfectGrey '  TJf hadllKT^Urhsi  of'liiiStKShiatict: aii v,  ;-s  2 

Bruck,  R.  H.  [1951],  Finite  nets,  1.  Numerical  invariants,  Canadian 
Journal  of  Mathematics  3:  9  4-107. 

Bruck,  R.  H.  [1963],  Finite  nets,  11.  Uniqueness  and  embedding, 
Pacific  Jou#haTof  Mathematics  J3:  421-457. 

Eruck,  R.  H.  [  1963]  ,  What  is  a  loop?  In  "Studies  in  Modern  Algebra" 
(editor,  A.  A.  Albert),  Math.  Assoc.  Amer.  and  Prentice  Hall,  Englewood 


Cliffs,  New  Jersey,  pp.  59-99. 
Dembowskl,  P.  [19  68]  ,  Finite 
Heidelberg,  New  York. 


»,  Springer-Verlag,  Berlin, 


Euler,  L.  (1782)  ,  Recherches  sur  une  nouvelle  espece  des  quarres 
magiques,  Verh.  Zeeuwabh  Genoot.  VVetenschappen,  Vlissigen  9:  85-239. 
fisher,  R.  A.  and  Yates,  F.  [  19  57],  Statistical  Tables  for  Biological. 
Agricultural  and  Medical  Research.  5th  edition  < first  edition  in  1938), 
Oliver  and  Boyd,  Edinburgh  and  London. 

Golomb,  S.  W.  and  Posner,  E,  C.  [1964]  ,  Rook  domains,  Latin  squares, 
affine  planes,  and  error-distributing  codes,  IEEE  Transactions  on  Informa¬ 
tion  Theory  IT-10:  196-208. 


18.  Hall,  M.  f  19431,  Projective  planes,  Transactions  of  the  American  Mathe¬ 


matical  Society  54:  229-277. 

19.  Hall,  M.  Ir.  [1967]  ,  Combinatorial  Theory .  Blaisdell  Publishing  Co., 
Waltham,  Massachusetts,  Toronto  and  London. 

gretjpl, 


Pacific  Journal  of  Mathematics  5t  541-549, 

21.  Hedayat,  A.  [1968],  On  Singer  1-permutation,  Unpublished  paper, 
Biometrics  Unit,  Cornell  University,  Ithaca,  New  York. 

22.  Hedayat,  A.  [1969],  On  the  theory  of  the  existence,  non-existence,  and 
the  construction  of  mutually  orthogonal  I'-squares  and  latin  squares,  Ph.  D. 
Thesis,  Cornell  University,  Ithaca,  New  York. 

23.  Hedayat,  A.  [19  69]  ,  On  the  equivalence  of  a  set  of  mutually  orthogonal 
latin  squares  with  other  combinatorial  systems,  Paper  RM-237,  Dept,  of 
Statistics  and1  Probability,  Michigan  State  University,  East  Lansing, 

i  Michigan. 

24.  Hedayat,  A.  end  Federer,  W.  T.  [  1969]  ,  An  application  of  group  theory 
to  the  existence  and  non-existcncc  of  orthogonal  Latin  squares, 

Biometrika  56(3): 

25.  Hedayat,  A.  and  Federer,  W.  T.  f  19 69  |  ,  On  the  equivalence  ot  Mann's 
group  automorphism  method  of  constructing  an  0(n,  n-1)  set  and  Raktoe's 
collineation  method  of  constructing  a  balanced  set  of  f -restrictlonal  prime - 
powered  lattice  designs,  (unpublished  paper),  Biometrics  Unit,  Cornell 
University,  Ithaca,  New  York. 


26.  Hedayat,  A.  and  S*>id?n,  E.  [1969}  ,  On  d  method  of  sum  composition  of 
orthogonal  latin  squares.  Paper  No.  RM-238,  Dept,  of  Statistics  and 
Probability,  Michigan  State  University,  East  Lansing,  Michigan. 

27.  Hedayat,  A.  And  Seiden,  E.  (1969]  ,  Some  contributions  to  the  theory  of 
F*squaies,  Unpublished  paper,  Cornell  University  and  Michigan  State 
University. 

28.  Johnson,  D.  M.  ,  Dulmadge,  A.  L.  ,  and  Mendelsohn,  N.  S,  (1961],  Ortho- 
morphisms  of  groups  and  orthogonal  Latin  Squares  I,  ,  Canadian  Journal  of 
Mathematics  13:  356-372. 

29.  Kempthorne,  O.  [1952],  The  Design  and  Analysis  of  Experiments.  John 
Wiley  and  Sons,  Inc.  ,  New  .  ork. 

to.  Levi,  r.  W.  [1942],  finite  Geometrical  Systems.  University  of  Calcutta, 
Calcutta,  India. 

31.  MacNelsh,  H.  f.  [1922],  Euler's  squares,  Annals  of  Mathematics  23: 
.’21-227. 

12.  Mann.  H.  3,.  1 1942]  ,  The  construction  of  orthogonal  Latin  squares,  Annals 
of  Mathematical  Statistics  13:  418-423. 

3  3.  Mann,  H.  B.  [  1943],  On  the  construction  of  sets  of  orthogonal  Latin 
squares.  The  Annals  of  Mathematical  Statistics  14:  401-414, 

34.  Mann.  H.  B.  [1944],  On  orthogonal  Latin  squares ,  Bulletin  of  the 
American  Mathematical  Society  5U:  249-257. 

35.  Mann,  H.  B.  led.  )  1 1968]  ,  Error  Correcting  Codes  .  John  Wiley  and  Sons , 
Inc.  ,  New  York,  London,  Sydney,  Toronto,  pp.  xii  +  2  31, 


36.  Paige,  L.  J.  [1951],  Complete  mappings  of  finite  groups ,  Pacific  Jbumal 
of  Mathematics  1:  111-116, 

37.  Paige,  L.  I.  and  Tompkins,  C.  B.  [i960],  The  size  of  the  10  x  10  ortho¬ 
gonal  la  tin  square  problem,  Proceedings  of  the  Tenth  Symposium  in 

.'.r 

38.  Parker,  E.  T.  [  1959]  ,  Construction  of  some  sets  of  mutually  orthogonal 
latin  squares.  Proceedings  of  the  American  Mathematics  Society  10: 

946-94^. 

39.  Parker,  E.  T.  [1962],  On  orthogonal  latin  squares,  Proceedings  of 
Symposia  in  Pure  Mathematics,  American  Mathematics  Society  6t  43-46. 

40.  Peterson,  W.  W.  [1961],  Error- Correcting  Codes.  John  Wiley  and  Sons, 
Inc.  .  New  \  ork. 

41.  Haktoe,  B.  L.  [1967],  Application  of  cyclic  collineations  to  the  construc¬ 
tion  of  balanced  /-restrictional  prime  powered  lattice  designs,  Annals  of 

Mathematical  Statistics  38:  1127-1141. 

* 

42.  Raktoe,  S.  L.  [1969],  Combining  elements  from  finite  fields  In  mixed 
factorials,  The  Annals  of  Mathematical  Statistics  40:  498-504, 

43.  Raktoe,  B.  L.  and  Tederer,  W,  T,  11969],  On  irregular  fractions  of  an  sm 
factorial,  Unpublished  paper,  Biometrics  Unit,  Cornell  University, 

Ithaca,  New  ,  ork. 

44.  Ryser.  H.  I.  [1963],  Combinatorial  Mathematics.  The  Carus  Mathematical 
Monographs  No.  14,  Mathematical  Association  of  America  and  John  Wiley 
and  Sons',  Inc. ,  New  York. 


t 


\ 


m - 


45.  Sade,  A.  (1958],  GroupWas  orthogonaux,  Publicationes  Mathematlcae. 
5:  229-240. 

46.  Silverman,  R.  (I960],  A  metrlzation  for  power  sets  with  applications  to 
combinatorial  analysis,  Canadian  Journal  of  Mathematics  12j  158-176. 

4,7^-.-  Sinaer-^y^O  l^TTrA-clarara 

American  Mathematical  Monthly  67:  235-240, 

4h.  dt ovens ,  W.  L,  {low).  The  completely  orthogonaiized  Latin  squares. 

Annals  oi  Eugenics  9:  8  3-oj. 

,  i  -.■*—*** 

4 ■»,  Tarry.  G.  {1899J,  Sur  le  probleme  d'Euier  des  n  offlciers,  L'Inter- 

** 

mcdiaire  des  Mathematicians  6:  251-252. 


r 


ON  CONFIDENCE  LIMITS  FOR  THE  PERFORMANCE 
OF  A  SYSTEM  WHEN  FEW  FAILURES  ARE  ENCOUNTERED  , 

Sam  C.  Saunders 

Boeing  Scientific  Research  Laboratories 
Seattle,  Washington 

aasamblles »  are  bo  reliable  that  no  failures  are  observed  within  the 
time  available  for  testing.  This  can  pose  a  problem  in  both  interpreta¬ 
tion  and  analysis.  We  consider  here  the  oreblem  of  determining  lower 
confidence  bounds  on  the  reliability  of  a  complex  system,  such  as  the 
Saturn  1-C,  when  each  component  is  assumed  to  have  an  exponential  life 
and  different  components  have  different  multiplicities  within  the  system. 

We  discuss  the  assumptions  necessary  to  obtain  cottfidsnct  limits  using 
the  likelihood  of  the  data  when  only  a  few  failures  are  encountered.  The 
bounds  resulting  from  two  models  are  compered.  The  first  model  Is 
Bayesian  with  uniform  prior  distribution  of  the  failure  rates.  The 
second  model  regarda  the  failure  rates  virtually  as  unknown  constants. 

Hers  Che  argument  is  made  that  modela  of  the  first  type  are  deficient 
in  several  regarda  in  comparison  with  ths  second. 

0.  INTRODUCTION.  The  problem  of  determining  the  probability  of 
successful  operation  of  a  large  complex  eastern  whan  one  ha*  data  only  on 
the  reliability  of  the  components  bee,  over  the  past  decade,  been  the  subject 
of  many  investigations.  However,  much  of  the  literature  was  of  a  proprietary 
nature  end  wag  never  published,  for  axample,  see  [1],  [2)  and  the  references 
there. 

Some  of  the  studies,  see  [7]  and  [9],  were  based  on  an  asymptotic 
theory  for  which  the  precision  of  the  approximation  is  unknown.  Currently, 
much  of  the  analysis  is  baaed  on  Bayesian  methods  utilicing  subjective  prior 
assumptions,  see  [12]  and  [13]. 


Because  estimating  the  probability  of  failure  under  some  model* 
requires  that  at  least  one  failure  be  observed,  the  statistician  may  bs 
placed  in  the  uncomfortable  position  of  having  less  confidence  in  his 
estimates  of  reliability  when  fewer  failures  obtain.  Ultimately,  whan  the 
system  becomes  near  perfect  and  no  failures  are  obaerved  the  statistician 
has  no  confidence  if  his  procedures  are  necessarily  based  on  failure  analysis. 
In  this  unsatisfactory  situation  it  is  an  understandable  reaction  of  persons 
with  good  engineering  judgment  and  statistical  intuition,  to  form  a  distrust 
of  statistical  inference  and  its  "numerologlsts ,"  see  [3],  Some  recent 
surveys  have  been  made  to  determine  the  moat  useful  end  applicable  procedures 
for  current  needs.  One  of  the  most  comprehensive  is  [8]. 


►‘N-  II  % 
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In  this  note  we  examine  some  statistical  techniques  which  do 
not  depend  upon  the  sampling  method  yat  era  applicable  whan  there  is 
a  paucity  of  observed  failures  among  the  components  which  have  been 
tested.  The  archtypical  situation  for  this  study  will  be  ithe  Saturn 
1-C  and  the  data  which  was  available  prior  to  the  first  launch,  as 
given  in  Table  1. 


The  remainder  of  this  articls  has  been  reproducsd  photographically 
from  the  author's  manuscript. 


1.  THE  BASIC  MODEL 


Consider  a  complex  system  designed  to  perform  in  a  specified 
manner  when  all  of  its  components,  grouped  in  assemblies,  are 
1  opaiatt^  i  s  ^hev‘  conf  idefee  :;  Ve c  an  ha v«  -  that  ‘ -ilM&ll----  =  : : 

perform  adequately  for  a  specified  time. 

We  shall  consider  the  reliability  of  the  system  for  a 
specified  time  t  as  being  given  by 

(1.1)  h^.t)  •  expj-  ^ 

where  the  u>^  ere  known  weighting  factors  for  the  unknown  parameters 

t 

&  ■  .,XB).  The  problem  is  obtaining  a  lower  confidence  bound  on 

(1.1)  from  the  limited  amount  of  data  about  the  for  1-1,..., m. 

The  particular  form  (1.1)  can  arise  In  several  ways.  The  first 
we  mention  is  that  it  is  itself  a  lower  bound  on  system  reliability. 

th 

If  the  time  until  failure  of  the  1  component  is  exponential 

with  unknown  hazard  rate  X^  and  these  m  Components  are  in  a 

coherent  system,  then  there  exists  a  set  of  integers  q^,...,qo, 

th 

where  q.  ie  the  multiplicity  of  the  i  component  within  the 
m 

system  of  order  £  •  which  can  be  used  to  obtain  a  lower  bound 

1  1 

on  the  reliability  of  the  system  at  any  time  t  >  0.  This  lower 
bound  is  h^.t)  ss  given  in  (1.1)  with  replaced  by  q^. 

For  a  proof  of  this  result,  see  [4]. 


>  f 


An  assembly  containing  several  components  can  malfunction  by 
having  different  components  fail,  e.g.,  a  pressure  system  can  either 
rupture  or  leak.  These  separate  ways  of  failure  are  called  "failure 
modes"  in  the  current  terminology,  however,  they  correspond  to 
malfunctions  within  th*  subsystem  end  need  not  necessarily  cause 
;^>.sysitia  ;f»iiat»,  SL-iL-  .  •’!  • . ~~  7^:'  ~.7.'~v:'"-T777 

We  assume  • 

1*  The  time  until  malfunction  in  each  mode  of  a  given  assembly 

has  constant  hazard  rate  and  all  are  independent. 

Suppose  we  separate  the  possible  modes  of  malfunction  for  each 
Assembly  into  mutually  ekcluaiva,  functionally  independent,"  classes 
labeling  the  time  until  malfunction  for  the  ith  mode  of  the  Jth 

assembly.  The  time  until  malfunction  of  the  assembly  by  any  mods  is 

TJ  “  “Jn<Tij)  *nd  th«  h««d  ot  Tj  1b  Xj  -  Ix1Jt  with  the 
obvious  interpretation  of  as  the  hazard  rats  of^  T  . 

Unfortunately,  the  system  may  have  diffarent  vulnerability  to  such 
malfunctions  depending  upon  the  period  within  the  mission  phase. 

Thus  the  second  situation  In  which  the  form  (1.1)  can  arias  1 
is  a  series  system  with  such  malfunctioning  assemblies.  Assume 

2*  Civen  a  malfunction  in  th«  Jth  mode  of  the  ith  assembly 
during  the  kth  time  interval  (t  ,t  of  amission, 
system  failure  will  reault  with  known  conditional  probability 
Sljk‘ 

For  a  mission  of  fixed  length  t  we  define  the  beta  factor 
for  the  1th  assembly,  which  is  a  known  constant,  by  the  equation 

•?  -r, 


8i  -  I  I811k[t(k)-t(k“l)] 

1  J.k  J 

where  t^  ■  0,  lim  t^  ■  t.  Thus  from  1*,  2*  the  probability 

.  '<'1 

‘'^eyfttw-ief Itv--- bver-'^tRi-- ellaAfttsrvai  (0,t)  la 
in  (1.1)  with  u.t  replaced  by  fl. .  Without  lose  of  generality  we 

X  L  i 

•hell  henceforth  assume  that  all  time  intervals  are  expressed  in 
fractions  of  this  fixed  mission  length  t,  to  wit,  assume  t  ■  1. 

Needless  to  say,  in  the  practical  example  given,  ell  of  these 
complications  occur  simultaneously.  Moreover,  the  determination  of 
the  through  minimal  state  reliability  analysis  is  itself  a 
non-trivial  task,  not  to  speak  of  the  analysis  of  the  malfunction 
modes  and  their  effect  upon  mission  success  (or  vehicle  safety). 

All  of  this  is  a  necessary  prelude  for  the  determination  of  the 
beta  factors.  But  ve  phall  assume  this  work  has  been  completed 
•o  that  is  known,  consequently  equation,  (1.1)  holds 

with  t  ■  1. 

The  problem  that  we  wish  to  discuss  is  of  another  genre, 
namely  the  methods  for  utilisation  of  the  data  so  as  to  determine 
a  lower  confidence  bound  for  the  reliability  after  the  have 
been  obtained.  Since  It  is  this  aspect  which  is  important  we  shall 
assume  that  =  1  for  our  data  so  as  to  further  emphasize  the 
differences  between  the  two  models  under  discussion.  Of  course  this 
fictitiously  makes  the  reliability  estimate  low. 


2 .  THE  DATA 


In  many  cases  tl'.i  first  (and  sometimes  the  only)  data  one  has 
concerning  the  reliability  of  the  component*  cornea  from  environmental 
teats.  This  test  data  must  be  reduced  by  engineering  evaluation  into 
the  equivalent  operational  time  during  the  given  mission  phase. 

practice,  see  JiJ,  to  accelerate  the  testing  and/or  reduce  its  expense. 
Specifically,  during  the  first  phase  of  a  mission  a  component  may 
experience  several  types  of  vibration  as  well  as  several  temperature 
apd  humidity  changes.  Consequently,  testing  the  components  in  these 
separate  environments  must  yield  results  requiring  a  transformation 

into  the  appropriate  mission  phase  equivalent  time.  (The  dangers  of 

*  ^ 
such  s  procedure  are  apparent  but  are  taken  in  view  of  the  axhorbltant 

cost  of  the  alternative.) 

We  do  not  discuss  this  further  but  we  merely  point  out  that  in  ^ 
such  Instances  the  data  on  the  operational  behavior  of  the  components 
•re  not  Cod-given,  but  rather  are  Che  construct  of  engineering  knowledge 
and  judgment. 

Thus,  the  statistician  is  ultimately  provided  with  data  on  all 
components  in  the  fora 

(2.1)  •  (t1,ni)  1-1,. ..,m 

where  t^  is  the  total  time,  expressed  in  equivalent  fractions  of  the 
mission  length  that  the  iC^  component  has  bean  operated,  and  n^  la  the 
total  number  of  malfunctions  of  the  l^*1  component  during  time  t^. 


.[ 


I 


i 

In  the  life  qualification  of  components,  it  is  usually  the 
case  that  testing  proceeds  as  long  as  there  are  funds  available 
and  this  Is  usually  neither  until  a  fixed  number  of  hours  nor  a 
fixed  number  of  failures  occur.  Moreover,  there  are  always 
extraneous  circumstances  which  might  terminate  the  testing  program 


*hy."titt*.  '.In: view  oft  hi*  in^etannihacy  in  the  sampling  pj*it6dufe 


the  treatment  of  the  daft  that  we  adopt  should  not  depend  heavily 
upon  a  particular  Sampling  scheme  which  might  not  obtain. 


Assume  that  a  number  of  identical  components,  say  m,  are  put 
on  test.  What  is  observed  at  each  trial  is  the  random  length  of 
life,  call  it  Y,  when  the  component  fails  or  the  random  time,  say 
Z,  at  which  the  test  Is  terminated  for  any  reason  other  than  failure 
of  the  component. 

Thus  we  observe  the^event 

\ 

[Y  -  y]  n  [V  <  Z]  or  [Z  -  a]  n  [Y  Z] 

which  we  note  is  not  the  minimum  of  Y  and  Z  since  we  know  whether 
Y  or  Z  is  observed.  It  is  well  known  that* the  likelihood  is  independent 
of  the  sampling  method  for  type  I  or  type  II  censoring,  l.e.  stopping  at 
.either  a  fixed  number  of  failures  or  after  a  fixed  time". 


We  make  the 

Remark :  If  (Z^,..*,Z  )  ia  a  vector  of  non-negative  random  variables, 
independent  of  all  Y4  such  that  [Y^  >  Z±] ,  where  (Y1P...,Y  )  are 
themselves  independently  and  identically  distributed  with  common  density 
function  f  and  distribution  F,  then  the  likelihood  of  the  event 


-09 


-V&r 


n 


n  IV.  <  Z.][Y 
i-i  1  “  1 


i 


n 

i“n+l 


IV£  >  Z{]  [Zt 


where  ft  m)  1#  the  random  number  of  failures  observed,  Is  of 
the  form 


(2.2) 


Jp 


and  the  constant  C  depends  upon  the  outcomes  (y^, ... *yn)  but 
not  upon  their  distribution. 


The  proof  is  immediate.  Let  g  be  conditional  density  of 

(Z, Z  )  given  (Y, , ...,Y _)  assuming  independent  of 

l  m  in  i  ,  ■ 

i 

Y  on  /~V  [Y,  »  2. ].  Then  the  probability  of  the  event 

0+1  “  i-n+1  11 

specified  above  is  , 


jCTi  . '.|yi . r»>d*i . 


which  upon  simplification  shows,  that 


C  ■  *[  •  •  • 

lal,...,n} 

claimed. | | 


/*<«! . *,1*1*  •  •  a  t  •  •  • 


Taking  the  data  x4  ■  (t^.n^)  for  the  ith  of  a  assemblies,  the 
1th  assembly  having  exponential  life  with  hasard  rate  Ij,  and  substituting 
into  (2.2)  we  obtain  the  likelihood 


(2.3)  pCxJlj)  •  C(x1)A11s  1  1 

where  C(Xj.)  is  independent  of  A^. 

Because  the  likelihood  ie  the  same  for  this  very  general  sampling 
situation  and  we  feel  that  the  data  by  its  nature  requires  such  independence 


■•W  (  \ 


vc  favor  methods  of  smHuMciit  analysis  which  depend  upon  the 
likelihood. 


From  Bayesian  Principles,  Lindley  [8],  pp.  1,2,  the  joint 
poitefiori  density  of  X  *  (A,,...,X  ),  based  on  the  evidence 

^  i  SI 


(2.4) 


f&lj£)  ^TT  P^I^MX.) 


where  n  is  the  Joint  prior  density  of  K 

Two  difficulties  remain.  One  is  to  formulate  a  reasonable 
joint  prior  ir  and  the  second  is  then  to  calculate, ! other  than 
symbolically,  the  posterior  distribution  of 


(2.5) 


V  ■  IBiX1,  say  G(v|ji)  for  v  >  0. 


Following  the  usual  method,  p.  15,  Lindley,  loc.  clt.,  the  value 
Vq,  depending  upon  x  and  e,  0  <  e  <  1  such  that  Q(Vq|;c)  -  c, 
provides  a  lower  lOOc^  Bayesian  confidence  bound  for  the  system 


-v. 


-V  To 

reliability  e  given  of  the  form  e  and 


(2.6) 


-v„ 


P[exp{-r8iA1}  >  e  w]  ■  c. 


Essentially  this  method  has  been  utilized  to  obtain  confidence 
bounda  on  the  reliability  of  certain  systems  and  la  presently  the 
subject  of  much  discussion,  see  [8].  In  what  follows  we  shall 
discuss  two  such  methods  and  their  reasonableness  in  dealing  with 
the  situation  at  hand. 


3.  A  UNIFORM  PRIOR 


The  first  approach  is  to  assume  the  special  prior  density 
<3 .1)  tr  s  1  for  all  At  >  0. 

of  insufficient 

reason:  since  we  know  nothing  specific  about  ir  ve  have  insufficient 
reason  to  take  n  anything  but  uniform.  Strictly  speaking  ir  as 
defined  in  (3.1)  is  a  non-probabilistic  prior.  But,  of  course,  one 
could  consider  it  proportional  to  an  approximation  to  a  prior  density. 

i 

Substituting  (3.1)  into  (2.4)  we  find 


mU - s7r— 


The  mathematical  problem  becomes  that  of  finding  tha  distribution  of 
V  ■  where  8^  are  known  constants  and  art  gamma  varlataa 

with  known  scale  and  shape  parameters.  To  wit,  each  Aj  '  If  rU^.n^+l) 
where  r(t,v)  denotes  the  law  with  density,  given  v  >  0 


(3.3) 


for  x  >  0. 


We  also  quote  two  related  results,  see  p.  46ff,  Feller  [7]. 

If  A^  is  r(t^,ui),  then  t^Aj  is  r(l,Vj). 

If  Vj  >  1,  then  Aj  -  Aj  +  A”  in  distribution  where 

is  r (t j ,  1 )  independent  of  A^1  which  is  r(tj,v^-l). 

Thus  by  the  first  remark  we  see  that  in  distribution  V  ■  §  b.A. 

J-l  J  3 


where  bj  -  6j/tj  ■  1/tj,  J"1 . m  eac^  Aj  *“  *low  "(l.flj+l). 

By  the  second  remark,  for  the  data  given  in  Table  1  where  we  have  at 
most  two  failures,  we  see  that  In  distribution 


where  s  is  the  numbeir  of  components  with  two  failures  during  testing 

* 

r-s  is  the  number  of  components  with  one  failure  during  testing 
m-r-s  is  the  number  of  components  with  no  failures  during  testing 


and  X^(  Vj ,  are  all  Independent  r(l,l),  i.e.,  exponential  with  unit 

mean,  variates. 

) 

We  now  quote  a  result  proved,  for  example,  in  [11]  as  a 

ianffij.:  If  li . zk  .r.  lnd.p.nH.nc  .xpon.ntul  r.adcm  v.tl.bl..  »lth 

unit  ms an,  then  for  >  0,  all  distinct,  we  have 


P[  i  b1Z1  >  u]  -  £  B(k)a"U/bj 
1  11  j-1  3 


where  B^  »  1  end  for  k  >_  2 


BaW- 


k  b. 


nvh 

t+i 


for  J*1| » *  # |k* 


Also  these  recursion  relations  hold 


Bjk>  ■  Bjk"1>bi/(bj“V»  J"1 . .  and  Bkt>  ■ 1  ■  *2  Bj° 


Also  we  have 


m  r 

Lemma  2:  The  distribution  of  2.  *• 


f»'i  ■»  »> 


But  the  quantity  in  braces  in  the  equation  above  becomes 


-Ut. 


fu  - (u-v)f . 

!•••}  -  Vi(u)-e  “Ti  L  Vi(v)e  dv- 


Hence  ve  have  shown  the  following: 

Lemma' 5  :  fhe  lutvlWr^robaWirihy  oT"  V^  af  defined  in"¥^(ifftiefi'  (3  ;  7)T 

is  given  in  terms  of  the  survival  probability  of  as 

%  (\) 

(3.8)  F  (u)  -IB* 

*  i-1 


"UT  . 


+  T 


-(u-v)t . 

Fk-l(v)e  dv 


Mote  that  (3.8)  can  be  used  to  prove  Lemma  2  and  used  recursively 
to  find  the  distribution  of  for  small  k.  Thus  we  now  have  a 
computationally  feasible  method  for  the  calculation  of  the  distribution 


of  V,  called  G,  Using  the  lemmas  above  a  machine  program  was  written 

(k) 

for  the  IBM  360,  using  double  precision  for  the  computation  of  the  Bj  , 
which  tabulates  the  distribution  of  V  in  the  region  of  interest.  Uaing 
the  data  presented  In  fable  I,  this  distribution  is  graphed  in  Figure  1. 

For  example,  we  find  that  if  vQ  -  12.8,  G (v0>  -  .95.  Thus  a 

j 

lower  95%  Bayesian  confidence  limit  for  the  system  reliability  is 
e-12’8  i  IQ"5. 


Figure  J. 


Graph  of  the  distribution  F  of  the  random  variable  V  -  EB^X^ 
in  the  region  near  unity.  If  v  it  the  abaciasa  value,  the  value 
F(v)  of  the  ordinate  ia  the  confidence  the  eyatem  reliability 
exceeds  e”v. 


4.  A  CRITICAL.  BISC'«?inN 

A  word  about  the  computation  necessitated  by  this  method.  It  1b 

clear  from  Table  1  that  the  differences  of  the  tj  are  neither  small 

(k) 

nor  uniform.  A  glance  at  the  formula  for  the  Bj  In  equation  (3.6) 
jSit--  in  absolufiL  value  they  can  become  AreTy  Targe  for  such  cases.. 

(In  fact,  for  such  data  as  we  have  for  67  components,  values  as  high  as 

20  (k) 

10  sre  not  Impossible.)  Since  all  Bj1  summed  over  j  must  add 

to  unity,  some  must  be  positive  and  some  negative.  However,  because 

of  the  nature  of  machine  decimal  arithmetic,  the  summands  will  be 

rounded  off  and  the  machine  cumulate  the  error.  We  ahould,  by 

definition,  have  G(0)  ■  0  but  computationally  we  do  not.  For 

example,  referring  to  Figure  1,  Che  machine  value  at  v  -  3.2  waa 

C(3.2) '■  .699  *  10"3  but  at  v  -  2.4,  G(2.4)  -  -.123  *  10*1  with 

wider  fluctuations  for  smaller  values  of  v.  Fortunately,  we  are 

interested  in  those  valjuea  of  the  argument  for  which  •  G(v)  is  near 

one  and  tha  values  of  V  necaasarily  become  large  enough  to  eliminate 

the  errors  due  to  this  circumstance. 

However,  this  Is  merely  a  limitation  due  to  the  accuracy  of  the 
method  of  computation  which  was  adopted.  We  feel  there  is  a  much  more 
primary  objection. 

Without  any  real  loss  of  clarity  to  the  fundamental  ideas,  let 

us  fix  our  attention  on  an  assembly  with  two  separate  modes  of  mal- 

* 

function  with  hazard  rates  A ^  and  A2,  say.  Suppose  this  assembly 

was  operated  for  a  time  t  and  no  failure  of  either  type  wap  observed. 

•  * 

By  using  the  uniform  prior  of  (3.1)  the  posterior  distribution  of  the 
component  hazard  rata  A(  •  Aj+Aj)  considering  ^he  component 

unit,  is  ~ 

-Hi* 


as  a 


so  chat  e~*  is  a  lowdr  bound  on  Che  reliability  for  a  mission  of 
unit  length  and  the  confidence  level  is  P[X  <  a]. 


But  on  the  other  hand,  by  considering. the  posterior  distribution 
Insufficient  reason  to  apply  the  uniform  prior  for  each  mode,  we  have 


(4.2)  P(X  <  a]  -  1-e  at-ate  aC  for  a  >  0 

ft 

as  the  posterior  distribution  of  the  hasard  rate  X  of  the  assembly. 

(  But  notice  that  (4.2)  is  less  than  (4.1)  which  was  the  distribution 
from  the  same  data  for  the  same  assembly. 

The  point  we  are  making  is  simply  this:  For  a  series  system, 
with  no  component  failing  during  test  time  t  >  0,  the  confidence  in 
the  reliability  of  thf  system  should  be  the  same  as  that  for  each 
component,  since  the  system  and  the  components  both  experience  the  j 
same  operational  time  t  without  failures 

Our  criticism  of  the  former  method  is  that  the'  confidence  in  thi 
reliability  docs  not  depend  only  upon  the  data,  it  also  depends  upon 
the  arbitrary  designation  of  component  or  assembly.  If  wo  arrive  at 
different  answers  when  using  the  same  data,  then  something  must  be 

I 

wrong.  , 

To  continue  this  point  further,  let  us  suppose  that  we  have  a 
series  system  with  separate  malfunction  modes  with  hasard  rates 
X,,...,i  each  of  which  has  acquired  the  same  operational  experience, 

*  **  II 


namely  x.  ■  (t,  0)  for  1-1,. ..,m,  i.e.,  no  failures  during 

*  X 

operation  for  a  length  of  time  t.  Again  by  iising  the  uniform 
prior  density  we  have  the  distribution  of  1  *  Xj+*»*+XB  as 


which  approaches  zero  as  m  approaches  infinity  regardless  of 
(the  value  of,  ta  >  0. 

Of  course,  (4.2)  results  from  a  different  specification  of  the 

>  ; 

Bayesian  model. .  The  point  is  that  each  specification  of .  another 
independent  component  wijl  always  result  in  a  different  posterior 

t 

distribution.  (Needless  to  aayj  e  different  prior  will  lead  to  a 
different  posterior  density  for  X  as  veil.) 

-  o  v 

\ 

Moreover,  It  is  clear  that  almost  any  choice  of  prior- density \ 

4  \ 

■Off'  1  which  is  the  product  of  Independent  prior  densities  for  each 
X^  will  result  in  e  conlfidence  level  which  Is* essentially  the'  same 
se  that  given  In  (4.3),  to  wit  so  low  as  to  be  nonsensical  for  m 
„  large .  ' 

'  t 

One  modification  suggested  is  to  assume  functional  dependence 
with  statistical  independence,  among  the  prior  densities  of  X^, 

One,'  such  is  to  take  the  (conjugate)  prior  density  of  X^  as  r 
r(ut,vi)  for  i«l,...,m,  using  here  the  notation  of  (3.5), 
subject  to  the  constraint 


Combining  this  with  the  likelihood  of  the  fora  (2.3)  shows 


m  n.+v.-l  -A. (u.+t . ) 

fuix.) ^TT  ur  1  e  1  1  1  ] 

1=1 


subject  to  (4;4)  above.  Thus  the  posterior  density  of  _A^  is 


finding  the  distribution  of  0  »  where 


and  each  is  r(l,n^+Vj)  for  i«l . m  subject  to  (4.4). 


Because  is  not  an  integer  we  are  faced  with  an 


analytic  and  computational  problem  beyond  that  of  the  preceding 
section.  However*  it  is  clear  that  this  artlface  does  introduce 
enough  degrees  of  freedom  that  proper  choice  of  u ^ ,  maintaining 
the  restriction  (4.4),  can  yield  reliabilities  of  not  unreasonable 
size.  We  do  not  pursue  it  further.  The  difficulty,  making  such 
an  assumption  untenable,  is  that  ones  prior  knowledge  about  the 
rellabllltyyjDf  a  component  should  neither  depend  upon  the  prior 
densities  of  the  other  components  in  any  way  nor  upon  how  many 
of  them  there  are.  These  prior  densities  should  be  independent 


5.  THt  DfcGI NKRATK  f’K'OR 


Ant  1/0310  ui  uo;t 


random  variables  having  a  distribution  which  is  to  be  constructed 
from  prior  knowledge.  The  more  commonly  accepted  point  of  view 
is  that  the  are  unknown  constants  about  which  inference  must 


If  the  >.^  were  unknovm  positive  real  numbers,  theft  there 
would  exist  a  constant  of  proportionality  between  any  two  \Js 
which  would  be  fixed,  even  though  it  was  unknown. 

Thus  we  make  the  assumption  . 

3*  There  exists  a  constant  of  proportionality,  say  ct^, 
between  any  two  A^  and  Aj. 

If  we  have  m  different  modes  of  malfunction,  we  define 

fch 

for  i«l,.  , ,  ,m  as  the  probability  of  malfunction  in  the  i  mode 
given  that  a  malfunction  in  the  system  has  occurred.  One  sees  that 


m 

o.  -  P[T  <  t|  X  [T  <  t] ] 
1  1  j-1  3 


where  we  made  the  convention  that  the  summation  of  events  denotes 
the  disjoint  union.  It  follows  that  a,,  *  u./a  where 

iJ  1  j 
^i 

(5.0.1)  aj  “  - - 


1  m 

1  * 

j“l 


i 


Thus  3*  is  equivalent  to  taking  the  prior  distribution,  say 
to  be  singular  with  all  measure  concentrated  along  a  ray  out 
from  the  origin  with  the  direction  of  the  ray  determined  by  the 
constants  of  proportionality.  Specifically,  we  assume 


(5-1)  dK  (A 4  •  •  •  *  * X  ) 

1  B 


a  A 

>  0  if  some  for  i  4  \ 

i  0j 


0  otherwise. 


In  the  case  m  *  2,  n&^Xj)  zero  everywhere  but  alon&  the 

*a'y.  ^  ouc  from  the  origin  In  the  plane;  We  wish 

to  find  the  posterior  density  of  ][  0.A  .  We  make  the  change  of 

1  1  1 

variables  P{--  and  by  (2.3)  and  ^  -  ti/fi±  ve  have 


where 


Thus 


(tiPl>nie  TlPidn*(p, . p  > 

i«l  *  ® 


P1  pm 

it*(p^, . . . ,pn)  ■  n(g  > ••••^  ). 

1  n 


(5.2)  dn*(Pl,...,p  ) 

X  m 


ai6iei 

>0  if  some  p4  -  for  i  4  j 


i  ajSj 


■  0  otherwise. 


The  density  we  seek  is  proportional  to 


(5.3) 


/  •••  /fT  (TiPi^e  TiCldne(p 

l£,:Epi“a}  1-1  1 


Consider  the  line  in  m-space' 


a-B,  a  6 

Z(pi)  “  (pi*  ^b^  pi . ^1 Pi>' 


By  equation  (S..>)  all  the  of  *  is  t  imo  n!  MK'd  •llont>  thv  r  •'* 

*  (.  \  fnr  „  -v  n.  In  effect  the  only  quantity  that  has  a  distribution 

'  1'  "  1 

in  p3  and  we  shall  later  see  It  makes  no  difference  what  this 
distribution  is  as  long  as  it  has  support  on  (0,®).  This  line  inter- 

B 

sects  the  plane  5!  p  *  a  at  a  single  point,  namely  Oj  such  that 
1 

“  0 p srt — 

1  i-2  °r l 


and  solving  for  we  find  p^  ■  where  we  define 


(5.3.1) 


m 

I  M. 


i-1 . m. 


Then  the  value  of  p±  at  the  point  of  intersection  of  the  line  1  with 
the  plane  a  is  ■  ay^  *or  i“l , . . •  ,m.  Since  all  the  measure 

of  n*  is  concentrated  along  the  line  l ,  the  integration  over  the  plane 
in  (5.3)  yields  a  single  value  at  the  singularity  of  the  measure  II*.  It 
follows  that  the  density  we  seek  is  proportional  to  the  value  of  the 
integrand  at  that  point,  namely 


m  n.  m 

TT  (T.avJ  exp{-  l  Tjavj). 

i-1  11  1-1 


If  we  define 


a 

0  *  I 


Etiai 

ZBiai 


k  -  I  n  , 
1 


;  «  > 
-  t_- 


and  the  total 


w!;  iJi  ar»  ,  respectively,  .1  weighted  mean  of  the 
number  of  failures,  we  can  write  the  posterior  density  of  V 
as 


f 

1 


Pi 


<5.5^ 


ki 


for 


a  >  0* 


the  distribution  then  is 


(5.6)  P[V  <  u] 


L 


0u  k  -s 
s  e 


k! 


ds 


1-  5 

j“0 


k  ~0u ,  .j 
e  (9u)J 


which  we  recognize  as  a  Chi-square  distribution.  If  we  set 
(5.7)  u  *  29  Xc<2k+2>* 

j  at 

where  x* (®)  i*  the  100c  percentile  of  the  Chi-square  distribution 
with  a  degrees  of  freedom,  we  have  e  u  providing  a  lower  confidence 
bound  of  level  c. 

We  note  that  the  computation  for  this  method  Is  trivial.  We 
compute  only  the  two  quantities  k  and  0  and  then  from  a  table  of 
the  Chi-square  distribution  calculate  u  and  e"11. 

Unfortunately,  equation  (5.7)  gives  the  confidence  bound  e  U 
in  terms  of  the  alpha  factors  which  are  still  unknown.  Nonetheless, 
based  upon  the  objective  model  that  the  failure  rates  are  virtually 
unknown  constants,  we  do  arrive  at  (5.7)  and  knowledge  concerning 
the  is  what  is  needed  to  determine  the  confidence  limit. 

However,  this  does  not  necessarily  mean  that  the  values  of 


need  be  known,  fot  example,  it  Is  sufficient  th.il  their  ratios 
be  known.  Perhaps  in  some  instances  engineer  ing  cjipei  l^uCC  alight 

be  able  to  classify  all  the  failure  rates  as  multiples  of  fixed 
one,  say  the  lowest,  at  least  in  a  conservative  manner. 

Disregarding  for  Che  present  the  computation  of  9,  this 
method  does  obviate,  some  of  the  conceptual  difficulties  which  the  , :  ••  JZ "V* 

preceding  method  possessed. 

Firstly,  the  confidence  bound  is  the  same  regardless  of  how 
the  components  are  apportioned  to  subsystems  within  the  system. 

In  particular.  If  ■•••■  t^,  we  obtain  the  same  density  of 
as  we  would  by  considering  the  system  as  a  single  unit. 

The  addition  of  components  to  the  system  none  of  which  have 
failed,  i.e.,  data  of  the  type  (t^.O),  do  not  necessarily  cause 
the  confidence  to  go  rapidly  to  zero.  (Of  course,  the  confidence 

does  depend  upon  t±  through  0,)  ,-It  is  clear  from  (5.5)  that  it 

/  ' 

la  not  the  number  of  components  but  the  number  of  failures  which 
rapidly  decrease  the  confidence,  1 

In  the  special  case  when  f,^  *  Bj  for  all  j  ,k  we  can 
^aake  an  intuitive  interpretation  of  as  the  conditional 

probability  of  failure  of  the  i^  assembly  given  that  an  assembly 
has  failed.  To  see  this,  label  the  events  "the  assembly  fails" 
by  and  "the  iLh  assembly  malfunctions"  by  Mj.  By  definition 

6i  *  p(FilMi].  -  P[Mi!EMj] 


( 


ar.d  from  the  calculus  r>f  lilies , 


P(F  ) 
al6i  *  P[ J 

and  hence  from  (5.3.1)  follows  *  PfFjjEF^J.  ( 

obtained  by  this  method. 


Example  1: 

Let  us  suppose  that  y.^  ■  ~  for  1*1,..., m.  We  recall  that 
under  certain  conditions  this  would  mean  the  event  any  one  particular 
component  had  failed,  knowing  that  exactly  one  component  wee  in  a 
failed  atate,  waa  equally  likely  with  the  event  any  other  component 
had  failed. 


From  Table  I  we  find  k  *  8  and  compute  from  (9.4),  G  ■  Et^/m  - 
25.35  and  hence  for  c  ■  .95,  tising'the  Chi-square  value  for  16 
degrees  of  freedom,  we  have  u  *  (28.87)/50.7  -  .569.  Thu*  a"11  - 
.566  is  a  lower  952  confidence  limit  for  the  system  reliability. 

Example  2; 

Let  us  suppose  ~  for  i*l,...,a  and  from  Table  I,  we 

again  use  (9.4)  to  compute  <?  *  (Et^/fEB^)  *  16.27.  For  k  *  8, 

e  -  .95  we  find  u  -  (28.87)/(32.54)  -  .887  and  e”u  •  .412  is 

( 

the  lower  95%  confidence  limit  for  the  reliability  of  the  system. 


m 


6.  BOUNDS  ON  9 


In  this  section  we  make  the  argument  that  what  prior  Information 
one  has  about  X^  for  i-l,...,m  should  be  applied  so  aa  to  determine 
bounds  on  e  rather  than  in  the  production  of  prior  distributions  of 


t  failure  rates. 


s  clear  that  if  g  “  (a^*.**,am)  is  constrained  and  a  lower 


bound  eL  <  fl(g)  can  be  determined,  then  correspondingly  from  (5.7) 

"“l 

u,  >  u,  from  which  it  follows  that  e  provides  a  lower  confidence 
bound  of  level  not  leaa  than  c.  For  example,  the  trivial  inequality 


■in  x4  <  6(a) 

1-1  1  “ 

will  provide  such  a  bound.  However,  unless  the  are  nearly  all 
equal,  a  state  devoutly  to  be  wished  and  planned  for,  it  is  not  certain 
this  bound  would  be  a  useful  result.  However,  if  testing  were  continued 
until  -  Tq  for  1-1,..., m,  we  would  then  be  in  the  favorable 
position  that  naad  not  be  known. 

If  tj  were  the  minimum  of  -r^  for  1-1 . .  then  -  6(g) 

implies  Yj  "  1»  *  0  for  1  1  j  which  in  turn  by  (5.3.1)  Implies 

that  oj  -  1*  -  0  for  i  i  J  which  requires  by  (5.0.1)  that  X^ 

be  Infinitely  large  with  respect  to  all  other  X^.  This  would  seem  to 
be  an  unlikely  state  of  nature,  one  which  might  be  reasonably  excluded 
from  consideration. 

We  now  give  some  examples  of  information  which  in  various  dsgrees 
exclude  the  state  mentioned  above  and  are  of  a  type  which  may  provide 
a  non- trivial  bound. 


Let  us  suppose  it  is  known  that 


m 

(6-1)  £  048,  *  P- 

i-1  1  1 

(This  would  mean  In  certain  situations  that  the  probability  of  a 

8 

failure  given  a  malfunction  was  known  to  be  p.) 

as  defined  in  (5.4)  where  (t^,^)  are  known  positive  numbers  subject 
to  the  restriction  (6.1)  and 

a 

(6.2)  £  a,  ■  1,  a.  >  0  for  l«l,...,m. 

1  1  1  ” 

Call  the  set  of  £  satisfying  (6.1)  and  (6.2)  the  set  efp. 

Clearly  with  the  denominator  fixed  in  (5.4)  we  have  a  linear 
programming  problem  with  two  constraints,  for  which  the  theory  is 
well  known. 

Of  course  the  restriction  (6.1)  is  a  mathematical  convenience. 
What  ve  desire  are  bounds  on  min  ^(p)  for  p  taken  over  some 

p 

subset  of  the  range 

min  B.  <  p  <  max  6^, 

where 

(6.3)  iKp)  •*  min{0(a>):  £  e  0(p}. 

This  can  be  obtained  from  a  graph  of  ^(p),  which  is  here 
accomplished  with  a  linear  program  using  p  as  a  parameter.  A  plot 
using  the  data  of  Table  I  is  given  in  Figure  2  as  on  illustration. 


/ 


Consider  0  as  a  function  of  y  <■  (ylt...,Ym)  as  defined  in 

(5.4).  It  is  linear  (hence  convex)  and  we  wish  to  minimize  it  subject 

to  some  restrictions  on  its  domain  which  are  measures  of  the  variability 

of  the  for  1*1, First  we  shall  consider  the  region  $ 

1  -1  * 


defined  for  6  <_  *  <_  r'm-1 


(6.4)  1  <  ml v?  <  2 


-  ““’i 


<_  x  +  1,  IYj  ■  1*  y<  t.  0  f or  1*1,..., m. 


The  region  $  is  convex  and  thus  there  exists  a  unique  minimum 

for  $  over  the  region.  The  method  we  shall  use  is  Lagrange  multipliers. 


Let 


mX.  « 

*(y)  -  s(£)  -  “2  Zyl  -  ^2zyt. 


We  wish  to  minimize  $  subject  to  the  conditions 


I  J 

!  i 


(6.5)  mTy2  -  x2  +  1,  Zyi  *  1,  y4 


>  0  for  1*1, 


Thus 


We  now  consider  the  three  equations 


■"  1 


I  1 


(6.6)  ly,  £•--(>.  Jim  -0, 

i  3'(i  “  1  hj  n 

which  upon  simplication,  and  imposing  the  restrictions  of  (6.5), 
yield  three  equations  v.'hich  are  to  be  solved  for  6  by  eliminating 

X,  and 


1 


'(  ' 


y  -  l*- a  r-'ir 


fli 


First  eliminating  \ ^  we  obtain 

(6.7)  X  -  sL.  ,  (e^)2  .  x2a2 

1  0-? 

where  t  and  a  are  the  mean  and  standard  deviation  of  i^’a, 
respectively.  -  •  .  ...  .... 

^^liii^il^ett-value  of  '  x  -  we  dan  d^tel^tbS^  I"Li5 

of  6  from  (6.7)  as  the  smaller  root 

(6.8)  0  ■  t  -  x a. 

Also  from  *  0  we  oust  have 

■  ■  ,yj 

(6.9)  Yj  ■  A^1m(Tj-X2)  >.  0  for  j-l,...,n 

in  order  to  satisfy  the  restrictions.  Since  by  (6.7)  end  (6.8) 

*  -o/x  we  see  that  a  sufficient  condition  to  satisfy  (6.9)  is 

(6.10)  max(Tj)  <_  Xj  ■  t  +  ox"*. 

This:. is  satisfied  for  all  t.  which  are  reasonably  close  together. 

It  iaelaar  from  (6.8)  that  the  minimum  value  of  0  is  a  decreasing 
function  of  x. 

Remark:  If  satisfy  (6.10)  for  a  given  x  between 

0  <  x  <  /m-l ,  then 

min{0(^):  %  c  -  t  -  xo 

where  t,o  are  defined  in  (6.7). 

From  this  remark  we  see  the  restriction  (6.4)  yields  a  lower 
lOOel  confidence  bound  for  the  reliability,  namely 


* 


.  .  /  ■*  a 


.0.1)  exp{-;^(2k+2)/2(7-xj)}  for  0  <_  x  <_  of  (max  t  )  • 

Using  thl*  data  in  Tabic  I  we  find  i  =  25.35,  o  =  21.25  and 
max  -  =  98.1  and  thus  the  range  of  x  is  0  <_  x  .31.  A  graph 

'  j 

of  (6.10.1)  for  this  case  with  k  =  7,  E  =  .95  is  given  in  Figure  3. 
Next  we  consider  the  problem  of  minimizing  9  subject  to 


(6.11)  =  <, 


=  1,  >_  0  for  i-1 , . . .  ,ra. 


Since  we  can  write 


a.  =  Yj3T1/(Z'fisT1)  fov  j  =  l,...,m, 

2-2  -12 

the  first  restriction  can  be  put  in  the  alternate  form  )  ■ 

Again  we  use  Lagrange  multipliers  to  take  advantage  of  the  symmetry  of 
the  problem.  Write 


mA 


r  -  yiYi-n 


It  -  Y  -  -  y  ■ 


-lx«-l, 


1  1  S.^0.  i  V  6^=0,  V  T.f^-0,  I  x,Bi 

”  n  1  3  °Yj  1  J  3Yj 


1  j  j  3Yj 


We  now  consider  the  four  equations 

}2  ii_  =  o. 

^  i  ’  R1  ^  ’  1  3v  .  -  1  i  n Y  . 

1  J  !j 
By  setting  • 

imposing  tin-  restraints  as  encountered  we  obtain  four  equations 

which  contain  the  four  variables  '  ^  *  2  ’  °  ’ 5  ’  Eliminating  and  6 

a  q-a  •  irat  i  c  equa.t  ion  in  r  ,  namely 


v  -  *  =  (Zi.3.)  ^  for  notational  convenience  and 

jj  J-'1 


■am r 


.2. 

v  A  r  r  C  *  U 


E 


where,  recalling  ^  for  i  »  1,  ...,m  , 


A  -  (I)2  -  xg2,  B  -  t S  -  xtg,  C  -  (t)2  -  xt2 


end  ali  coefficients  depend  upon  Che  value  of  x  «  1  -  (auc)“^ 
Thus  the  minimum  value  1$  the  smaller  root,  call  It 


(6.14) 


S*  “  (Bx  *  V/Ax 


where  S  ■  B2  -  AC  or  equivalently  S2  -  xa2  -  x2b2 


and 


*2  *  m  l(tl®"eJ)2»  b2  "  *V  -  («>2. 


It  la  clear  that  if  x  is  constrained  by 


(6.14, i)  0  <  x  < 


min 


S&L  E  J&l  ,  1 

e2  CB  t2  2b2  m 


then  the  values  of  9X  are  meaningful. 

We  now  argue  that  there  is  an  interval  of  positive  values  of  ■  x 
for  which  ©x  is  decreasing  as  a  function  cf  x.  Note  9'  <  0  iff 


(6.15)  4(x)  -  -S^Ax  -?S  s«  (g)2  - 


MPT 


To  see  that  c  ia  an  increasing  function,  note  that  c'(x)  “  . 


Now  S'(x)  »  (a^-2xb2)/2S(x)  so  that  by  (6.14.1),  S*(x)  >  0  and 


likewise  we  check  S"(x)  <  0  and  hence  c'(x)  2  0,;  Notice  that 
;(0)  *  -«  so  there  is  an  interval  of  values  in  xj  for  which  (6.15) 


is  true,  and  9x  is  decreasing.  This  region  can  be  determined  from 


However,  we  must  also  satisfy  the  condition  Yj  *.  0  for 


j->l,...,m.  From 


m 


»Y< 


0  follows,  by  using  (6.13) 


(6.16) 


Yj  >  0 


iff 


t  8,e-.,  »(t,-6,(e)) 


-  <mXj6 


t-86 


But  by  the  argument  above  6^  <  0q  »  t/0  for  x  >  0.  Hence  the 


denominator  of  the  right-hand  side  of  (6.16)  is  positive.  Thus 
>_  0  for  j-l,...,m  iff 

m 


(6.17) 


t  ■  e  J>x  oax{t.-e  s,}. 

J*1  3  3 


To  be  presuaded  that  there  is  indeed  a  neighborhood  of  zero  in  which 


(6.17)  is  true  we  Introduce  the  power  series  expansion  of  $x: 


9-1  -Vx  +  0(x). 


(6) 


Thus  (6.17)  is  equivalent  with 


2 

J.  _  +  Q(x)  >  /*  max[t.-9  6.] 

i  (I) 2  “  j  j  x  3 


-msl 


fcwT*lS!i  *rfr|’'r>  '  f 


which  is  clearly  true^Tor  x  sufficiently  small.  Thus  we  can  make 


Remark i  If  (t^,p^)  i-1,...,®  are  given  and  x  satisfies  (6.14.1),'-, 


then  3X  defined  by  (6.14)  satisfies 


CONCLUDING  REMARKS.  By  considering  a  series  system  with  a  given 
number  of  components,  each  component  having  been  tested  for  the  same 
time  and  each  experiencing  no  failures,  one  intuitively  feels  since  the 
entire  system  could  have  been  operated,  conceptually  at  least,  without 
failure  for  that  same  period  of  time  that  the  confidence  one  has  in  the 
system's  reliability  should  be  exactly  the  same  as  the  confidence  in 
each  component's  reliability.  Moreover,  this  confidence  should  be  the 
same  irrespective  of  the  number  of  components  in  the  system. 

By  thinking  how  the  confidence  should  behave  for  such  a  series 
system,  as  components  are  added  with  different  test  times  and  different 
numbers  of  failures,  we  see  that  the  Bayesian  approach  with  independent 
prior  distributions  of  the  failure  rate  fails  to  fulfill  our  expectation 
as  to  this  incremental  behavior.  At  the  same  time  the  second  model  chosen, 
with  failure  rates  as  virtual  constants,  does  seem  to  behave  in  conformity 
with  our  intuition  and  moreover  it  has  the  added  appeal  of  computational 
simplicity. 

Lastly,  for  the  practical  case  chosen  namely  early  data  for  the 
Saturn  1-C  which  at  this  juncture  we  know  is  a  highly  reliable  system, 
the  second  model  gave  reasonable  interval  estimates  of  the  reliability 
while  the  first  did  not. 
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A  PROBABILITY  MODEL  FOR  THE  ASSESSMENT  OF  HUMAN 
INCAPACITATION  FROM  PENETRATING  MISSILE  WOUNDS 


William  P.  Johnson 
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Aberdeen  Proving  Ground,  Maryland 


ABSTRACT.  A  mathematical  model  is  proposed  for  the  evaluation 
of  the  altered  performance  of  one  of  the  most  complex  systems  known 
to  man,  himself.  Probabilities  are  associated  with  a  random  fragment 
penetrating  varying  distances  within  the  human  body,  striking  a  critical 
anatomical  component  and  inflicting  damage  to  the  extent  that  the  wound 
recipient  would  be  unable  to  perform  his  assigned  duties. 

These  probabilities  are  combined  to  determine  the  conditional 
probability  of  all  events  occurring,  simultaneously,  to  a  tactical 
soldier  under  battlefield  conditions. 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  authors'  manuscript. 
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1.  INTRODUCTION 


To  evaluate  the  effectiveness  of  antipersonnel  weapons  it  h as 
become  necessary  to  employ  a  quantitative  casualty  criterion,  A  suit¬ 
able  criterion  for  many  purposes  is  the  probability  that  the  weapon 
will  wound,  fatally  or  severely,  its  Intended  target. 


j|n  old  criterion  of  wounding  power  is  the  58  foot  pound  rule,  In 
its  crudest  form  Xt.Ji.alcajUta^oifeftUos  .with  -iefrs  -fcl^ 

Jlo  -n ,  =5^r=¥oo t  s 


of  kinetic  energy  do  kill.  This  criterion  was  never  intended  to  be  more 

1# 

than  a  rough  rule  of  thumb.  Rurns  and  Zuekerman  wmadc  a  more  refined 

analysis  in  1941  of  the  quantitative  requirements  for  wounding,  while 
2  ,  ’ 

Gucrncy  suggested  that  MVJ  was  a  more  suitable  criterion  than  the 
kinetic  energy  standard  for  wounding  human  targets.  . 


Wound  ballistics  work  carried  out  in  this  country  during  and  since 
World  War  II  has  included  studies  of  the  relationship  between  tho  volumes 
of  cavities  formed  in  tissues  and  tissue  simolants  by  missiles  and  the  ^ 
physical  parameters  describing  the  projectile  on  imjiact  with  the  target. 

Such  work  ha£  not  yet  provided  numerical  relations  between  the  physical 

i 

parameters  of  fragments  and  the  probability  that  the  woilnded  man  will 
suffer  any  specified  degree  of  impairment  of  his  ability  to  function. 

For  the  evaluation  of  the  antipersonnel  weapons,  there  is  needed  a  know¬ 
ledge  of  the  numerical  probability  that  a  man,  struck^by  a  projectile  of  , 
specified  characteristics,  will  thereafter  be  unable  to  perform  the  func¬ 
tions  of  his  tactical  role.  For  evaluation  or  design,  the  size  of  a 
wound  /is  not  directly  employable  as  a  useful  criterion.  So  fiir,  cavity 
studies  have  unfortunately  led  to  little  or  no  information  suitable  for 
the  evaluation  or  design  of  fragmenting  weapons. 


The  Hums  and  Zuekerman  studies  led  to  numerical  probabilities  of 
the  type  needed  for  evaluation.  A  later  numerical  study  of  a  similar 
type  was  reported  by  McMillcn  and  Gregg  of  the  Princeton  Department  of 
Biology,  in  Missile  Casualties  Report  No.  12,  6  Nov  1945,  National 
Research  Council,  Division  of  Medical  Sciences.,  McMillcn  and  Gregg  were 
concerned  with  wounds  that  thev  considered  to  be  either  fatal  or  severe;  ... 

_  ■  )  "V"  1 

_  V  ,-j 

Surcnccrint  numeral.-  refer  to  references. 
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These  could  be  caused,  they  assumed,  by  projectiles  reaching  certain 
vulnerable  regions  inside  the  body  after  traversing  protective  layers 
of  skin,  soft  tissue,  and  boric  provided  that  the  projectiles  reach  the 
vulnerable  regions  with  velocities  in  excess  of  750  m/sec.  The  thick¬ 
nesses  of  the  anatomical  structures  that  had  first  to  be  traversed  were 
determined  from  anatomical  charts  depicting  cross  sections  of  the  body 

head  to  feet.  The  velocities 


were  determined  through  in  vivo  experimentation.  The  selection  of 
vulnerable  regions  was  somewhat  arbitrary^  since  no  experiments  were 
conducted  to  determine  which  regions  were  in  fact  of  primary  importance. 
The  final  results,  which  were  therefore  semi-theoretical  in  nature,  were 
in  the  form  of  probabilities  that  random  hits-with  steel  balls  on  a 
human  target  would  cause  fatal  or  serious  wounds.  These  probabilities 
were  plotted  against  the  striking  energies  of  the  balls,  and  were  on  the 
whole  consistent  with  the  earlier  conclusions  o'f’Uurns  and  Zuckerman. 


A  generalization  of  quite  a  different  typo  was  published  by  T.  H. 
Sterne  in  May  1951.  In  that  studj^  the  experimental  data  employed  by^> 
McMillcn  and  Gregg  wore  re-examined  and  re- interpreted.  In  addition, 
the  calculations  of  McMil len -and  Gregg  were  repeated  for  randomly  shaped 
fragments  similar  to  bomb  fragmonts  instead  of  spheres.  The  temporary 
cavities  caused  by  the  fragments  penetrating  the  target  were  taken  into 
account  by  requiring  that  a  fragment,  in  order  to  cause  important  injury 
to  a  vital  region,  must  not  only  penetrate  the  intervening  skin,  tissue, 
and  bone,  but  must  also  reach  the  vulnerable  regions  with  sufficient 
remaining  energy  to  produce  a  temporary  cavity  of  2  cubic  centimeters. 


In  October  of  1956,  "A  New  Casualty  Criterion  for  Wounding  by  1 

4  I 

Fragments"  was  published  by  Allen  and  Sperrazza.  This  study  revised, 

the  combination  of  mass  and  velocity  combinations  which  appeared  to  be 

related  to  incapacitation.  Instead  of  employing  the  MV/A  parameter  used 
6 

by  Sterne,  MV  ,  where  1  <  6  <  2,  was  introduced  for  the  first  time.  This 
criterion  has  been  used  extensively  to  determine  the  relationship  between 
trauma  and  fragment  characteristics.  Howfevcj:,  cloie-examination  of  the 
techniques  used  within  the  current  model  reveals/ that  the  resultant 
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quantitative  data  is  based  upon  numerous  subjective  assumptions  which 
vary  from  one  evaluation  to  the  next  and  that  the  resultant  data  are 
not  probabilistic  numbers  in  the  mathematical  sense. 

With  the  advantage  of  the  knowledge  available  in  the  area  from 
earlier  investigations,  it  is  the  purpose  of  the  current  study  to  estab¬ 
lish  a  model  that  will  supersede  the  currently  used  fragment  criteria. 

The  specific  objectives  of  the  proposed  study  are: 

(1)  To  establish  a  truly  probabilistic  model  for  the  assess¬ 
ment  of  human  incapacitation  from  penetrating  missile  wounds. 

(2)  To  eliminate  the  necessity  for  tissue  retardation  firings 
and  in  vivo  experimentation  as  a  prerequisite  for  the  evaluation  of  the 
wounding  potential  of  future  fragments. 

(3)  To  eliminate  the  need  for  trajectory  tracings  through 
anatomical  cross  sections  of  the  human  body  in  order  to  establish  quanti¬ 
tative  values  for  the  probability  of  incapacitation. 

/  (4)  To  establish  a  common  basis  for  the  comparison  of  all 

fragmenting  munitions. 

The  approach  and  procedure  for  establishing  the  proposed  model  while 
accomplishing  the  above  objectives,  arc  presented  in  the  following  sections 
of  this  paper. 

II.  APPROACHES  AND  ASSUMPTIONS 

For  the  purposes  of  establishing  a  mathematical  model  to  predict  the 
probability  of  incapacitation  to  3  human  upon  impact  from  a  random  pro¬ 
jectile  the  following  assumptions  arc  made: 

(1J  The  fragmenting  pattern  of  grenades  and  other  exploding 
munitions  have  been  studied  in  sufficient  detail  to  provide  the  proba¬ 
bility  of  a  specific  fragment  striking  a  human  target,  P(H). 

(2)  Each  fragment  impacting  the  human  target  has  a  distinct 
probability  of  penetrating  specific  distances  within  the  wound  recipient's 
body  ,  I’ O'). 


I 
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(3)  Each  fragment  striking  the  casualty  has  a  distinct  proba¬ 
bility  of  encountering  a  critical  organ  along  its  path,  P(E).  This 
probability  is  conditioned  by  the  penetrating  ability  of  the  fragment 
and  the  location  of  the  critical  components  of  the  human  anatomy. 

(4)  Each^anatomical  component  has  a  distinct  probability  of 
being  damaged,  P(D),  to  che  extent  of  preventing  specific  biomechanical 
motions  required  by  a  tactical  soldier  for  the  full  performance  of  his 
mission. 

(5)  Each  specific  biomechanical  motion  has  a  distinct  proba¬ 
bility,  P(M) ,  of  being  required  during  the  performance  of  the  soldiers 
total  mission. 

(6)  The  probability  of  incapacitation  from  the  N'th,  P(N), 
component  may  be  described  by  the  expression: 

P(N)  =  P  (H)  •  P(P/H)  •  P  (E/HP)  •  P  (D/HPE)  •  P(M/HPED) 

(7)  Each  of  the  component  probabilities  are  independent  and 
can  be  combined  mathematically  to  provide  the  conditional  probability  of 
incapacitation,  P(I/HPEDM),  to  a  tactical  roldicr  from  a  random  projec¬ 
tile  by  the  following: 

N 

P(I/HPEDM)  =  1  P(N) 

1 

N 

For  the  purpose  of  brevity,  henceforth  the  conditional  probability  of 
incapacitation  of  a  tactical  soldier,  P(I/IIPEDM),  will  be  shortened  to 
P(I/H).  It  is  to  be  understood  that  the  second  expression  includes  all 
of  the  conditions  provided  within  the  first  expression. 

(8)  No  synergistic  effects  occur  from  multiple  wounds.  Thus, 
the  probability  of  incapacitation  to  a  tactical  soldier  from  two  or  more 
wounds  can  be  determined  by  mathematically  combining  the  independent  con¬ 
ditional  probabilities  associated  with  each  wound  using  the  following 
expression: 

P(I/»I)  =  1-(1-P  (i/ii1))(i--r’(i/ii2))(i-i’0/u3);)..... 
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As  mentioned  earlier,  the  probability  of  a  fragment  striking  a 
stationary  human  target  is  assumed  to  have  been  established  by  Exterior 
Ballistics  experts.  Therefore,  the  remainder  of  this  paper  will  con¬ 
centrate  on  the  procedures  to  be  used  to  establish  the  probabilities 
required  for  the  other  portions  of  the  model. 

III.  PROBABILITY,  P(P),  THAT  PROJECTILE  WILL  PENETRATE 
A  SPECIFIC  DISTANCE,  (D) ,  WITHIN  THE  HUMAN  BODY 

As  assumed  by  McMillan  and  Gregg,  it  seems  reasonable  to  suppose 
that  the  probability  that  a  random  hit  will  cause  fatal  or  severe  wound¬ 
ing  depends  upon  the  fraction  of  the  body's  superficial  area  through 
which  the  fragment  can  wound  a  vital  organ.  The  identification  of  the 
vital  regions,  and  the  conditions  of  striking  them  necessary  to  cause 
fatal  or  severe  wounding  may  not  have  been  correctly  chosen  by  McMillen 
and  Gregg,  nevertheless,  it  still  seems  reasonable  to  suppose  that  the 
probability  P(I/1I)  will  be  a  function  of  the  penetrating  ability  of  a 
fragment.  If  fragments  possess  such  great  penetrating  power  that  they 

'  J  ■ 

traverse  a  body  completely  wherever  they  hit,  then  the  probability  P(I/H) 
that  a  random  hit  will  incapacitate  is  the  ratio  of  a  rather  large  vulner¬ 
able  area  to  the  total  presented  area  of  the  body.  On  the  other  hand,  if 
the  penetrating  power  of  a  fragment  is  so  low  that  it  can  never  reach  the 
critical  components  of  the  body,  then  the  probability  P(I/H)  must  ap¬ 
proach  zero.^ 

It  is  proposed  that  an  indication  of  the  penetrating  potential  of 
fragments  be  obtained  thru  the  use  of  facilities  currently  or  soon  to  be 
available  to  wound  bal listicians.  These  include  (1)  the  Ballistic 
Research  Laboratories  Computer  Man^  and  (2)  striking  versus  residual 
velocity  comparisons  for  each  of  the  anatomical  components  of  the  human 
body. 

A  brief  diversion  is  required  at  this  point  to  acquaint  the  reader 
with  the  BRL  Computer  Man,  in  order  that  he  may  fully  appreciate  its 
potential  for  aiding  in  the  solution  of  the  current  problem. 

In  brief,  the  BRL  Computer  Man  is  a  computer  program  currently  used 
within  the  wound  ballistics  program  to  determine  the  extent  of  incapacita¬ 
tion  experienced  by  wounded  tactical  soldiers.  It  consists  of  coded 


versions  of  human  anatomical  cross  sections  extracted  from  Eycleshymer 
and  Schocmaker,  "A  Cross  Section  Anatomy."  The  combination  of  the 
cross  sections  represents  a  human  male  in  a  specified  tactical  position. 
Every  major  anatomical  component  illustrated  in  the  original  cross 
section  anatomy  has  been  coded  within  the  computer  version  in  approxi¬ 
mately  the  same  proportion  as  found  in  the  published  version. 

The  associated  instructions  accompanying  the  coded  cross  sections 
within  the  computer  model  permit  simulated  fragment  paths  to  be  traced 
through  the  individual  cross  sections  at  various  impact  angles.  Retar¬ 
dation  data  is  provided  for  each  of  the  anatomical  structures  and  is 
used  to  determine  the  velocity  loss  within  each  coded  component  as  a 
function  of  the  distance  traversed  by  the  fragment  through  the  individ¬ 
ual  components.  Thus  the  penetrating  ability  of  each  fragment  becomes 
a  function  of  (1)  striking  conditions  at  impact  upon  the  cross  section 
and  (2)  the  retarding  ability  of  the  anatomical  components  encountered 
along  its  path. 

Retardation  data  for  several  anatomical  components  has  been  used 
within  the  Wound  Ballistics  program  for  several  years.  However,  the 
data  in  existence  is  of  inadequate  quality  for  the  ultimate  solution  to 
our  current  problem.  Fortunately,  a  program  is  currently  being  con¬ 
ducted  by  Sturdivan  and  Thompson6  which  should  provide  the  necessary 
retardation  data  input  to  the  BRL  Computer  Man. 

Upon  completion  of  the  retardation  studies,  the  data  will  be  fed 
into  the  Computer  Man  model.  Maximum  distances  traveled  within  the  ana¬ 
tomical  model  will  be  determined  as  a  function  of  several  physical  para¬ 
meters  of  the  impinging  projectile  (mass,  velocity,  presented  area). 
Success  or  failure  for  each  trajectory  will  be  determined  by  its  ability 
to  penetrate  at  least  each  of  several  pre-specif ied  distances  within  the 
human  anatomy.  The  probability,  P(G),  of  a  particular  fragment-velocity 
combination  penetrating  at  least  each  of  the  pre-specified  distances 
within  an  anatomical  subdivision  will  be  the  ratio  of  the  total  number  of 
successes  at  that  distance  to  the  total  number  of  initiated  trajectories 
not  perforating  the  subdivision.  It  should  be  noted  that  the  denominator 
becomes  the  number  of  non-perforating  trajectories  rather  than  the  total 


number  of  trajectories  originated  within  a  subdivision.  This  becomes 
necessary  because  of  the  varying  distances  within  the  human  anatomy  at 
which  perforations  will  be  achieved  by  most  projectiles  because  of  the 
variations  in  tissue  structures  encountered  along  each  path.  In  addi¬ 
tion,  the  geometry  of  the  human  anatomy  iS«such  that  the  distance  avail¬ 
able  fd*  penetmion^bpanyvp^  /  V 

farther  from  the  center  of  the  body.  ■ 

If  we  let  P*  represent  the  probability  of  a  projectile  perforating 
the  human  anatomy,  then  (1-?')  represents  the  probability  that  the  pro¬ 
jectile  will  stop  within  the  human  body.  However,  it  is  desired  that 
probability  figures  be  determined  as  a  function  of  soveral  distances  ^ 

within  the  non-perforating  group.  The  probability,  P(P),  associated  with 
each  non-perforating  projectilo  penetrating  at  least  each  of  the  pre- 
spocifiad  distances  will  be  given  by  the  product  of  the  probability  of 
retention  of  the  fragment  within  the  human  anatomy,  (1-P'),  and  the 
probability  of  tho  fragment  reaching  the  required  distance,  P(G). 

Symbolically,  this  becomes:  P(P)  »  (1-P')  •  P(G) 

There  appears  to  bo  two  alternatives  for  considering  the  perforating 
projectiles : 

(1)  It  can  be  assumed  that  thd  penetration  pattern  of  tho  non¬ 
perforating  projectiles  are  representative  of  that  which  would  have  been 
displayed  by  tho  perforating  projectiles,  if  the  human  anatomy  was  such 
that  it  allowed  all  projectiles  to  penetrate  as  deeply  as  possible. 

Based  on  this  assumption,  the  penetration  pattern  as  a  function  of 
distance  would  be  represented  by  P(P)  and  the  perforating  projectile'  - 
paths  would  only  be  used  to  condition  the  probability  of  the  body  retaining 
an  impinging  projectile. 

(2)  It  con  be  assumed  that  the  perforating  fragment  paths  repre¬ 
sent  the  upper  limits  of  penetration  and  that  because  they  do  perforate, 
the  fragments  would  be  able  to  roach  any  desired  depth  within  the  human 
anatomy.  Based  on  this  assumption,  tho  penetration  pattern  as  a  function 
of  distance  would  be  represented  by  the  sum  of  P(P)  and  P*. 


As  of  this  writing  it  has  not  boon  dec  id  od  which  of  the  two  alter¬ 
natives  will  be  used  within  the  proposed  pwwlri .  However,  since  wc  are, 
aware  of  the  available  options,  it  is  felt  that  subsequent  discussions 
with  knowledgeable  personnel  will  provide  ins i ght  us  to  the  proper 
approach , 

Probability  data  or  the  above  typo  will  be  obtained  for  several 
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he  used  to  roTafc  file  /H'Aii;djl!fiEtjF^ ^  '6^“spec±^fi^n^ 
a  physical  parameter.  of  tho  'Jirojec.ti le.  Thus,  generalized  equations  can 
be  developed  which  will  provide  the  user  with  tho  probability  of  a 
particular  projectile  penetrating  a  specified  distance  within  a  human 
as  a  function  of  its  impact  point  on  the  anatomy  and  other  relevant 
physical  characteristics  of  the  impinging  projectile. 


IV.  PR011A1U1.TTY  OP  PRO.lRCTII.li  UNCOUNTPHINC  THE  N'TIl  ANATOMICAL 
COMPONENT  1,'URI Nil  ITS  PATH  ‘illKOllGli  THE  HUMAN  BODY,  P(E) 

If  it  is  assumed  that  a  projectile  will  impact  the  human  target 
randomly,  then  it  must  ho  assumed  that  the  path  of  the  projectile  through 
tho  body  will  be  random.  Although  the  proposed  model  is  designed  to 
ylold  primarily,  the  probability  of  incapacitation  from  a  single  pene¬ 
trating  projectile,  it  should  be  realized  that  an  almost  infinite  com¬ 
bination  of  organs  or  tissues  can  be  encountered  along  the  path  of  a 
singlo  wound,  l-'or  this  reason  one  must  deal  with  the  probability  of 
encountering  each  independent  anatomical  component  within  u  given  wound 
tract,  rather  than  combinations  of  traumatised  organs.  The  shielding 
effects  afforded  some  organs  by  their  surroundings,  combined  with  the 
orientation  of  tho  organs  to  specific  impact  angles  makes  the  ratio  of 
t.hclr  size  to  tho  total  body  nireu  an  inaccurate  measure  of  the  proba¬ 
bility  of  encountering  the  orgaft. 

In  uddition,  because  tho  geometry  of  most  anatomical  components  vary 
as  a  function  of  their  depth  within  the  anatomy,  probabilities  of  encoun¬ 
tering  each  structure  must  be  developed  as  a  function  of  the  obliquity 
angle  of  the  impinging  projectile  and  the  target  depth  within  the  human 
body.  However,  for  the  purposes  of  this  mode!,  final  probabilities  will 
bo  documented  as  u  function  of  penetration  distance  only.  Several 
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obliquity  angles  will  be  used  to  determine  the  probability  of  encoun¬ 
tering  each  structure  as  a  function  of  its  depth  within  the  human  ana¬ 
tomy  .  Hie  final  probability  figures  will  reflect  the  weighted  averages 
obtained  from  each  of  the  independent  analyses.  Not  only  will  this 
approach  reduce  the  amount  of  bookkeeping  involved  within  the  project, 
it  should  also  yield  probabilities  of  encountering  specific  organs  more 


representative  of  those  expected  from  a  truly  random  penetration. 

obtained  within  this  phase  of  the  model.  Imaginary  trajectories  will  be 
traced  through  the  coded  cross  sections  and  allowed  to  penetrate  specific 
distances  within  the  human  anatomy.  For  each  organ  or  tissue  under  con¬ 
sideration,  a  strike  upon  the  structure  within  a  specified  distance  will 
be  recorded  as  a  success.  The  probability  of  encountering  each  structure 
as  a  function  of  the  specified  distance  will  be  the  ratio  of  the  total 
number  of  recorded  successes  to  the  total  number  of  trajectories  traced 
through  the  anatomy.  The  above  will  be  documented  at  unit  intervals  for 
each  anatomical  component.  The  final  probabilistic  numbers  will  reflect 
the  chances  of  having  encountered  each  component  with  penetrations  up  to 
the  indicated  depth. 


Subsequent  additions  to  the  proposed  model  will  provide  insight  on 
the  effects  of  missile  "bite"  on  the  probability  of  encountering  compo¬ 
nents  within  the  human  anatomy.  This  physical  phenomena  considers  the 
fact  that  a  projectile  need  not  actually  strike  a  component  in  order  to 
damage  it.  Investigations  in  theso  areas  are  currently  underway  within 
tho  Woynd  Ballistics  program  and  once  these  data  are  available  in  quan¬ 
titative  form,  the  proposed  model  can  be  modified  to  include  this 
phenomena. 


Subsequent  acquisition  of  modern  anatomical  cross  sections  reflec¬ 
ting  the  internal  structure  sizes  of  soldiers  from  specific  military 
anthropometric  percentiles  should  allow  the  probabilistic  data  generated 
during  this  subportion  of  tho  model  to  be /generalized  as  a  function  of 
a  physical  measure  of  the  human  anatomy.  No  data  exists  of  this  type  at 
the  present  time,  therefore  it  is  impossible  to  make  such  generalizations 
or  even  speculate  as  to  how  the  probabilities  of  organ  encounter  are  ex¬ 
pected  to  vary  between  population  percentiles. 
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V.  PROBABILITY,  P(D),  THAT  DAMAGE  TO  AN  ANATOMICAL  COMPONENT  WILL 

PREVENT  THE  PERFORMANCE  OF  SPECIFIC  BIOMECHANICAL  MOTIONS 

Before  a  missile  wound  can  be  evaluated  in  terms  of  the  bio¬ 
mechanics  required  to  perform  a  given  task  or  role,  it  is  necessary  to 
describe  the  wound  in  terms  which  can  be  easily  related  to  the  phy¬ 
siological  function  of  the  body  or  subsystems  within  the  body.  That 
is,  criteria  must  be  developed  which  relates  the  probability  of  pro¬ 
ducing  a  given  damage  level  to  performance.  This  must  be  carried  out 
in  three  steps.  They  are: 

(1)  Determine  criteria  to  be  used  for  damage  description  of 
the  anatomical  components. 

(2)  Determine  the  probability  that  missiles  from  a  given 
munition  will  produce  a  specific  damage  level  to  a  particular  ana¬ 
tomical  component. 

-  * 

(3)  Determine  the  probability  that  this  damage  level  will 
result  in  some  biomechanical  decrement. 

At  present  work  is  being  conducted  under  area  (1).  This  task  is 
oriented  towards  wound  description  in  terms  which  can  be  related  to 
biomechanical  decrement.  The  criteria  used  to  describe  the  wound  will 
depend  on  the  particular  tissue  or  tissue  type  encountered.  For 
instance,  injury  to  muscles  may  be  described  in  terms  of  hole  size, 
percent  muscle  severed,  etc;  injury  to  blood  vessels  may  be  described 
in  terms  of  the  rate  of  blood  flow  from  the  injured  location.  Once 
these  criteria  have  been  determined,  step  (2)  can  be  undertaken. 

Presently,  there  exists  a  data  bank  of  wound  descriptions  through 
animal  tissue  in  terms  of  various  parameters  for  several  penetrating 
missiles.  If,  for  instance,  the  criteria  chosen  is  hole  size,  the 
wound  data  will  be  used  to  determine  the  distribution  of,  hole  size  as 
a  function  of  the  missile  parameters  (mass,  presented  area,  velocity). 
The  information  combined  with  the  distribution  of  missile  parameters 
for  a  given  munition  will  provide  the  distribution  of  hole  size  for 
that  munition.  Using  the  probability  distribution  for  hole  size  for  a 
given  munition  an  expected  hole  size  will  be  computed,  i.e.  the 
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expected  damage  level,  E(D).  Likewise,  if  the  criteria  for  damage  was 
blood  loss,  a  similar  E(D)  would  be  computed. 


This  value  of  E(D)  would  then  be  presented  to  a  medical  assessor 
who  would  subjectively  determine  the  probability  that  the  indicated 
damage  level  would  prevent  an  individual  from  accomplishing  a  specific 
biomechanical  motion. 


It  is  recognized  that  this  subjective  input  into  the 


events; 


in  order  to  quantify  the  effects  of  damage  levels  to  various,  anatomical ' 
components;  years  and  years  of  experimental  laboratory  work  would  be 
required.  As  of  this  writing  little  experimental  work  has  begun.  In 
view  of  the  fact  that  the  decision  made  by  the  medical  assessors  will 
be  a  function  of  the  expected  damage  to  alT anatomical  component,  totally 
independent  of  the  physical  parameters  of  the  impinging  projectile  and 


made  only  once  within  the  lifetime  of  the  modql ,  (except^ perhaps  to  im¬ 
prove  a  previous  decision),  it  is  felt  that  the  subjoctiVkity  will  be 
minimized  as  much  as  possible.  ) 


VI.  PROBABILITY,  P (M)V  THAT  A  SPECIFIC  BIOMECHANICAL  MOTION  WILL  BE 
REQUIRED  DURING  THE  PERFORMANCE  OP  A  SOLDIER'S  MISSION 

Due  to  the  wide  variety  of  specialties  among  servicemen,  it  is 
almost  impossible  to  generalize  upon  the  duties  of  active  combatants. 
However,  basic  to  the  performance  of  duties  associated  with  every  spe¬ 
cialty  is  tho  ability  of  the  individual  to  perform  controlled  movements 
or  biomechanical  motions.  Regardless  of  the  traumatized  anatomical 
components  or  region,  a  soldiers  ability  to  function  under  battlefield 
conditions  is  directly  related  to  his  ability  to  carry  out  controlled 

movements.  If  an  individual's  wound. site  is  such  that  it  does  not  ham- 
* 

per  or  impair  his  ability  to  make  the  necessary  biomechanical  motions, 
then  for  all  practical  purposes,  the  individual  cannot  be  regarded  as 
incapacitated.  These  controlled  movements  of  the  body  may  be  resolved 
into  the  functioning  of  the  following  subsections: 


I.  Shoulder  Girdle 


II. 

Shoulder  Joint 

III. 

Elbow  and  Radio-Ulnar  Joint 

IV, 

Wrist  and  Hand 

V. 

Pelvic  Girdle  and  Hip  Joint 

Knee-Joint 

.  •*-  - - .-L-i - — ?. :  .  •  .  "  j’  .  (| 

Vi!. 

VIII. 

Spinal  Column 

IX. 

Thorax 

Movement 

of  the  Shoulder  Girdle: 

1.  Adduction  -  movement  of  the  scapula  medially  toward  the  spinal 

column. 

2.  Abduction  -  sliding  of  the  scapula  laterally  and  forward  along 

the  surface  of  the  ribs. 

3.  Elevation/depression  -  the  upward  and  downward  motions  of  the 

whole  scapula  without  any  rotation. 

4.  Upward  rotation  -  involves  an  upward  turning  of  the  glenoid 

cavity  and  the  lateral  angle  in  relation  to  the 
superior  angle  and  medial  border,  which  turn 
downward,  '  , 

5.  Downward  rotation  -  reverse  of  upward  rotation. 

6.  Poreward  tilt  -  occurs  when  the  inferior  angle  moves  backward 

away  from  the  rib  cage. 

7.  Backward  tilt  -  the  inferior  angle  and  the  costal  surface 

return  to  the  surface  of  the  rib  cage. 

♦  . 

Movement  of  the  Shoulder  Joint; 

1.  Flexion  -  foreward  elevation  of  the  arm. 

2  Extension  -  return  movement. 

3.  Abduction  -  sideward  elevation  of  the  arm. 

4.  Adduction  -  return  movement. 

5.  Inward  rotation  -  turning  the  humerous  wound  its  long  axis  so 

that  its  anterior  aspect  moves  medially. 


- ' 


6.  Outward  rotation  -  the  opposite  with  the  anterior  aspect 
moving  laterally. 


A.  Elbow  Joint: 

1.  Flexion. 

2.  Extension. 

B.  Radio-Ulnar: 

1.  Pronntion. 

2.  Supination. 

IV.  Movements  of  the  Wrist  and  Hand: 

A.  Wrist  Joint:  ^ 

1.  Abduction.  / 

2.  Adduction. 

3.  Circumduction. 

4.  Flexion. 

5.  Extension. 

B.  Hand: 

1.  .JljMJhensile  movement. 

a.  power  grip  -  an  object  is  clamped  by  the  partly  flexed 

fingers  and  palm  with  counter  pressure  ap¬ 
plied  to  the  thumb  lying  more  or  less  in 
the  plane  of  the  palm. 

b.  precision  grip  -  object  is  pinched  between  the  fingers 

and  the  opposing  thumb. 

2.  Nonprehensile  movement  -  objects  are  manipulated  by  pushing 

or  lifting. 

V.  Movement  of  the  Pelvis  Girdle  and  Hip-Joint: 

A.  Pelvis: 

1.  Forward  rotation  -  increased  inclination  resulting  from 

lumbo-sacral  hyperextention. 

2.  Backward  rotation  -  opposite  movement. 

3.  I.atcral  tilt  -  the  lowering  or  raising  of  one  iliac  crest. 

4.  Rotation  -  turning  about  a  vertical  axis  either  to  the  right 

or  left. 


B.  Hip-Joint: 

1.  Flexion  -  forward  movement  of  the  femur. 

2.  Extension  -  reverse  movement. 

3.  Abduction  -  movement  of  one  limb  away  from  the  other  toward 

the  side, 

4.  Adduction  *  movement  of  one  limb  from  the  side  towards  the  other. 

5.  Circumduction  -  movement  of  the  limb  in  a  circular  manner, 

'Combine  movements  1-4. 

6.  Rotation  -  may  be  outward  or  inward  depending  on  which  way 

the  toes  are  turned. 

VI.  Movements  of  the  Knee  Joint: 

1.  Flexion. 

2.  Extension. 

3.  Inward  rotation.  / 

4.  Outward  rotation.  /  ' 

/ 

/ 

VII.  Movements  of  the  Ankle  and  Foot:  / 

/ 

A.  Ankle  and  Foot:  / 

1.  Dorsiflexion  -  consists  of  raising  the  foot  toward  the 

anterior  surface  of  the  leg. 

1  2.  Plantar  flexion  -  lowering  the  foot  so  ^s  to  bring  its  long 

axis  in  line  with  that  of  the  leg. 

3,  Eversion  -  the  solo  is  turned  laterally  or  outward. 

4,  Inversion  -  sole  is  turned  medially. 

B.  ToesN 

V 

1.  Flexion. 

2.  Extension, 

VIII.  Movements  of  the  Spinal  Coluntn: 

A.  Cervical  Spine: 

1.  Flexion. 

2.  Extension, 

3.  Lateral  flexion. 

4.  Rotation. 

il.  Thoracic  and  Lumbar  Spines: 


r 


1.  Flexion. 

2.  Extension. 

3.  Lateral  flexion. 

4.  Rotation. 

IX.  Movements  of  the  Thorax: 

1.  Elevation. 

2.  Depression. 

As  can  be  seen  from  the  above  the  possibility  of  a  wide  variety  of 
body  movements  exists  for  every  individual.  However,  we  are  concerned 
only  with  the  essential  or  necessary  movements  involved  in  the  tasks  of 
combatants.  1 

It  is  assumed  that  every  task,  regardless  of  its  complexity,  can 
be  resolved  into  a  series  of  controlled  movements  of  the  type  presented 
above.  It  is  further,  assumed  that  a  probabilistic  figure  which  reflects 
the  chances  of  an  individual  being  required  to  perform  a  specific  bio* 
mechanical  motion  during  the  course  of  his  duties  can  be  determined.  If 
an  individual  is  required  to  assume  multiple  duties,  then  the  probability 
of  his  performing  a  specific  biomechanical  motion  will  be  conditioned  by 
the  probability  of  his  performing  the  duty  which  requires  the  motion. 
Thus,  the  evaluation  of  a  soldier's  incapacitation  can  be  related  to 
several  duties  required  within  an  overall  mission. 

In  order  to  obtain  the  desired  probabilities,  one  must  be  knowledge¬ 
able  of  the  specific  duties  required  of  today's  soldier.  In  addition, 
these  duties  must  be  analyzed  from  the  kinesiologist  point  of  view  in 
order  to  reduce  them  to  thq,  scries  of  independent  biomechanical  motions 
required  for  probabilistic  analysis. 

Once  the  duties  have  been  reduced  to  a  series  of  controlled  move¬ 
ments,  two  alternatives  for  determining  the  probability,  P(M),  that  a 
specific  biomechanical  motion  will  be  required  during  the  performance  of 
a  soldier's  duties  or  mission  appear  to  exist.  The  first  assumes  that 
each  motion  is  independent  of  time  and  that  every  motion  involved  within 
a  specific  task  requires  the  same  percentage  of  the  sohlier's  time. 

This  docs  not  eliminate  the  possibility,  of  repetitions  of  the  samo 


movement  within  a  given  task.  Repetitions  are  accounted  for  by  includ¬ 
ing  each  repetition  as  a  separate  component  of  the  total  sum  of  move¬ 
ment?  involved.  The  probability.  P(M),  of  a  particular  motion  being 
required  within  the  task  will  be  the  ratio  of  the  total  repetitions  of 
the  movement  within  the  task  to  the  total  number  of  independent  move¬ 
ments  involved. 

The  second  approach  considers  the  total  time  required  by  the  com¬ 
batant  to  perform  his  duties  as  the  unit  and  the.  probability,  P(M),  is 
determined  by  the  functional  part  of  the  total  allocated  to  each 
Independent  motion. 

Each  of  the  two  approaches  (has  advantages  and  disadvantages  obvious 
to  the  authors  at  this  point.  However,  suffice  it  to  say  that  the  merits 
of  either  approach  will  be  discussed  in  detail  with  the  experts  on  the 
subject  prior  to  deciding  which  approach  is  most  feasible.  It  appears 
that  either  of  the  alternatives  will  yield  probabilistic  data  of  the 
type  required  for  input  into  the  model.  Therefore,  no  serious  problems 
appear  to  exist. 

VII.  SUMMARY 

A  mathematical  model  for  assessing  the  probability  of  human 
incapacitation  from  a  penetrating' missile  Wound  has  been  proposed.  The 
model  assumes  that  several  probabilistic  evcflts  must  occur,  sequentially, 
in  order  to  cause  incapacitation  to  a  tactical  soldier.  Among  these 
events  are  (1)  the  soldier  must  be  hit,  (2)  the  impacting  projectile  must 
penetrate  deep  enough  into  the  human  anatomy  lo  strike  a  critical  anatom¬ 
ical  component,  (3)  the  anatomical  component  must  be  damaged  to  the  ex¬ 
tent  that  its  physiological  function  is  impaired,  and  (4)  the  dysfunction 
of  the  component  must  bo  directly  related  to  the  soldier's  ability  to 
perform  specific  biomechanical  motions  required  for  the  performance  of 
the  soldier's  task. 

An  approach  has  been  given  for  obtaining  the  necessary  experimental 
data  needed  to  relate  each  of  the  above  events  in  a  probabilistic  manner. 
In  addition,  the  assumptions  and  anatomical  equations  used  within  the 
model  have  been  detailed. 


LOGIC  FOR  DETERMINING  THE  PROBABILITY  Of  INCAPACITATING 
A  SOLDIER  SUBJECTED  TO  A  8ATTLEFIELD  THREAT 


Q- 
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